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Preface 



For the twenty- fifth year, the Research and Theory Division of the Association for Educational 
Communications and Technology (AECT) is sponsoring the pubhcation of these Proceedings . This is 
Volume #2 of the 2 5‘'^ Annual Proceedings of Selected Papers On the Practice of Educational 
Communications and Technology Presented at The National Convention of the Association for 
Educational Communications and Technology held in Dallas, TX. A lim ited quantity of these 
Proceedings were printed and sold. It is also available on microfiche through the Educational Resources 
Clearinghouse (ERIC) system. 

This volume contains papers primarily dealing with instmction and training issues. Papers deahng with 
research and development are contained in the companion volume (25* Annual, Volume #1), which 
also contains over 60 papers. 

REFEREEING PROCESS: Papers selected for presentation at the AECT Convention and included in 
these Proceedings were subjected to a reviewing process. All references to authorship were removed 
from proposals before they were submitted to referees for review. Approximately fifty percent of the 
manuscripts submitted for consideration were selected for presentation at the convention and for 
pubhcation in these Proceedings . The papers contained in this document represent some of the most 
current thinking in educational communications and technology. 



M. R. Simonson 
Editor 
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Does It Translate? Promulgating the Revised AECT Code of Professional Ethics 

Worldwide 



Annette Sherry, University of Hawai'i 
Clarence Chu, National Taiwan University 
Candido Manuel Varela de Freitas, Universidade do Minho 
Jose Diaz de Rabago, S.J., Universidad de Santiago 
Abbas Johari, Cameron University 
Francois Marchessou, Universit_ de Poitiers 
R. W. Bumiske, University of HawaiT 



Abstract 

Educators from universities around the world examine challenges faced in translating the revised AECT 
Code of Professional Ethics into various languages. Issues related to dissemination of the Code at 
College of Education on a multicultural university campus are also explored. 

Translation is an imperfect science. In many instances, finding equivalent words in another language can 
be impossible. The situation is apt to be confounded even more when concepts that are generally accepted 
by one national or cultural group are unknown or puzzling in other cultures. 



Translation Project 

Despite such challenges, a team of educators from universities across the globe who were also members of 
the International Council (IC) of the Association for Educational Communications and Technology (AECT) initiated a 
process to disseminate the AECT Code of Professional Ethics worldwide. As the original Code was revised in 
November 2001 by that Committee and with the renewed emphasis in AECT on international issues, the time for 
providing translations appeared propitious. 

Participants 

Seven IC members began the initial process. Although this first group was formed on a voluntary basis, 
they do address five of the nine languages most frequently used on the Internet: English, Mandarin Chinese, 
Portuguese, Spanish and Erench (Global Reach, 2002), along with a less frequently used one, Earsi. Besides doing the 
actual translations, each translator was asked to reflect on the task of accurately expressing the essence of the 
concepts and terms of the Code in those languages. 



Code of Professional Ethics 
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The English version was examined in terms of ''translating'' the Code for a mass audience, that is for faculty, 
staff, and students at a College of Education at a Research I university. In this latter situation, graduate students in 
an educational technology program developed a display to focus attention on the new version of the Code that had 
expanded from 23 to 25 principles and includes modifications for five of the original ones. 

Those principles are expressed in three sections related to commitment to the individual, to society, and to 
the profession. Three principles follow to exemplify the expectations for professional behavior within these three 
categories. Each section begins with the phrase, “In fulfilling obligations to the individual, the members,” (AECT, 
2001, para. 5) and is followed by pertinent statements. Eor example: 

Commitment to the Individual. 1 .7. “Shall make reasonable efforts to protect the individual from conditions 
harmful to health and safety, including harmful conditions caused by technology itself.” (AECT, 2001, para. 5) 

Commitment to Society. 2.6. “Shall promote positive and, minimize negative environmental impacts of 
educational technology.” (AECT, 2001, para. 6) 

Commitment to the Profession. 3.3. “Shall avoid commercial exploitation of that person’s membership in the 
Association.” (AECT, 2001, para. 7) 

Dissemination 

Having a code is one step, promulgating it worldwide to members and others who become involved with 
instructional technologies is another. If the Code is to be disseminated globally, having an English-only version of 
the Code does not contribute to widespread usage. 



Spoken Languages Worldwide 



Considering that there are approximately 6,500 languages spoken by the 6 billion plus people in the world 
today, the task may seem insurmountable. Size becomes somewhat more manageable when it is realized that 2,000 of 
those languages are spoken by less than 1,000 people (Infoplease, 2001). The task becomes become infinitely more 
manageable when languages that prevail on the Internet are identified in conjunction with online choices of 
languages officially used at the Web site for the United Nations (UN). 

Language Usage on the Internet 

In the case of the former, it is estimated that nine languages are used by 3 percent or more of the online 
participants (Global Reach, 2002). In terms of the latter, four are selected by visitors to the UN Web site (United 
Nations, 2001) at a rate of 3 percent or more. As shown in Table 1, when those data are compared, the most 
frequently selected language is English. Although Chinese is not selected as frequently among visitors to the UN 
site, it is the next most frequently selected language based on global Internet usage figures. This ranking is not 
surprising, as Mandarin Chinese is the most popular spoken language in the world (Infoplease, 2001). It can be seen 
that percentages for use of Spanish and Erench on the Internet as a whole, as well as at the UN site are fairly similar. 

Table 1. 

Comparison of Languages Selected for Internet Use Globally and at the UN Web Site 



Language 



Percent of Global Internet Use 



Percent of UN Site Use 



English 


36.5 


84.6 


Chinese 


10.8 


0.5 


Japanese 


9.7 


0 


Spanish 


7.4 


8.0 


German 


6.6 


0 


Korean 


4.5 


0 


Italian 


3.8 


0 


Erench 


3.5 


4.7 
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Portuguese 



3.0 



0 



Note . Table is based on figures from Infoplease, 2001 and Global Reach, 2002. 

Initial Translation Team 

It was fortuitous, then that these initial translators who are educators in the field of educational technology, 
are fluent in Mandarin Chinese, Spanish, French, and Portuguese. Given that Arabic is one of the official languages 
of the UN, having a translator who has a Mideastern language, in this case, Farsi, as his first language reflects the 
spirit of the UN. 

Translators in the virtual world like their “real world’ counterparts are concerned with avoiding cultural loss 
and misunderstandings, providing choices, addressing concerns about equivalent concepts, and reaching broad 
audiences through their work. Many translators who prepare materials for the Web are also faced with avoiding 
products that result in “overweighf ’ Web sites, that is sites that exceed 40 K in recognition of slow connection 
speeds in many parts of the world (Yunker, 2002). This latter concern was not a problem in preparing the translations 
of the Code for the Web site of AECT, given its brief Preamble and 25 one-sentence statements. This team does, 
however, present some issues specifically related to their task, as well as concerns typically noted by translators. 

Mandarin Chinese Translation 

The translator for the Mandarin Chinese version of the Code, Clarence Chu from National Taiwan 
University, a native Chinese speaker, indicated that initially he thought the project would be easy. It was ultimately 
more difficult than he had expected, even though he is an experienced translator of many books 

He saw his task being exacerbated due to the similarity of the Code to a legal document. As such it requires 
great precision in choosing the right words with every sentence being translated in a very exact manner. For example, 
in the title alone, the word “Code” has six or seven different meanings in Chinese. Referring to a book of 
interpretative case studies based on the earlier version of the Code (Welliver, 2001), guided him as he worked to 
express the key concept behind each principle. Although Chu sees a need for translating that casebook into 
Mandarin as a further project, he also cites the need to localize some of the scenarios to make them more readily 
understandable for the targeted audience, thus, offering examples closer to their own cultural experiences. 

In Mandarin, it is not simply a matter of characters, but the meaning of the word itself. For example, the 
word, “technology” means “techniques” in Mainland China and something else in Taiwan. Not just the Mandarin 
characters, but also many scholarly terms are translated differently in both lands. For this reason Chu suggests that 
two versions may be necessary for the Mandarin Chinese translation. 

Portuguese Translation 

Candido Varela de Freitas from the Universidade do Minho, the native speaking Portuguese translator, also 
saw difficulties, noting that the Italians use a saying, “Tradutore, traditore,” which means a translator is a traitor. He 
sees the truth of this saying since at any step of his work he faced difficulties in putting into Portuguese concepts he 
had in English. 

Concepts cannot be translated, only words can, but words should be able to suggest the concept. The first 
dilemma he had to overcome, therefore, was “Should I do a literal translation or should I ‘adapt’ the Code to the 
Portuguese reality?” 

Although Varela de Ereitas thought that a literal translation would not be the ideal decision, an adaptation 
could become a problem because it could distort the real meaning of the Code. He, thus, began working to perform a 
literal translation, according the following procedures. He wrote a first draft and asked a colleague of his who has 
perfect command of English to revise his text. 

He also had a few meetings with colleagues from his university who work in the field of educational 
technology. In those meetings he explained the scope of his work and asked for contributions, mainly about how 
readable the text was. 

One of his colleagues found several statements “weird;" she could not understand what they meant. There 
was a consensus that although the objectives and ideas of the Code are understandable, the text itself made little 
sense to them as it was written. All of them, did however, recognize the need for such an instrument in Portugal, 
where each one working in the field of instructional technology has his or her ethics, but has nothing explicitly 
written. 
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Based on his work, Varela de Freitas identifies Principles 1.1, 1.5, 1.6, 2.2, 3.1, 3.3, 3.5, 3.9 and 3.10 for a 
thorough revision. 

Considering this situation, he thinks that the translation of the Code into Portuguese is important but that 
his current Portuguese version should be reformulated to more of an adaptation, rather than being basically a literal 
translation. The problem is that such adaptation has to be validated to become an official version of the AECT. 
Despite all these concerns, he keeps his interest in this challenging task. 

Farsi Translation 

Abbas Johari from Cameron University in Oklahoma, a native Farsi speaker who has worked in the United 
States for the past twenty -three years, concurs with the difficulties facing translators of this Code of Ethics, as 
evidenced by the time it took him. Flis original estimate of a two -hour process in actuality took three days. 

He was encouraged, however, when he saw that his colleagues had these problems, as he thought he was 
losing his language. Eor his translation he did not use traditional Earsi because he did not have that particular 
keyboard with him to create that version. 

He selected Earsi-Latin because it captures the sounds of the Persian language and is used by so many 
young people on the Internet. In that form of Earsi, the title of the Code and acknowledgement of the language used 
appears as, “AECT Kode wojdani: tarjomeye farsi.” In that way, the sounds of Earsi language are captured even 
though the traditional letters are not used. With 80 million people using Earsi, Johari sees the need for making such a 
version available, despite its not being perfect. 

Much like Varela de Ereitas, Johari sees the need to have like-minded colleagues in Iran polish it because of 
the potential for many different meanings. A particular challenge in his native language is that the word order is the 
opposite — progressing from right to left — as opposed to the left to right order of the English language. Eurthermore, 
the order of words within sentences is reversed. Eor example, in Earsi, the verb usually appears at the end of the 
sentence whereas in English it typically appears in the middle. 

Johari illustrated his painstaking translation process with a sample of the worksheet he used. Multiple 
meanings for each word were written and then examined to see what made the most sense. He went through 
numerous words to find the one that made sense in developing a usable version. 

He notes the worry and responsibility he feels and assures all that it will be reviewed by competent 
translators in the tourist industry in Iran and by university friends. 

Erench Translation 

Expressing his perspective, Erancois Marchessou from the Universite de Poitiers in Erance, a native Erench 
speaker with over forty years of experience doing translations, sees the Code as being too itemized. The Erench, he 
notes, would take much of it for granted. He sees its meanings as challenging. Eor example, translating “ethics” to the 
Erench “ethique” would be viewed as study in the philosophy department. “La morale” refers to ethics in everyday 
life; it is “built in;” it is the rules which govern your own life. Ethics is something that you practice, not something 
you publicize; it is personal. “Put it in a code, yes,” he says as he acknowledges that there has to be a code. 
Marchessou, thus, sees the need for a translation and recognizes the consequences, but cautions against using the 
more theoretical term, “ethique.” 

To further explicate national differences in terminology, consider that Americans may say that they do not 
like someone’s ethics, while the Erench would be more apt to say they do not like the person’s moral sense. Similar to 
the concern noted by Varela de Ereitas, Marchessou sees translation in some ways as being a betrayal or at times a 
poetic expression. 

Context and background become problems. Who do translators have in mind as they translate? Considering 
translations for French speaking peoples in developing countries where he frequently consults (as well as some 
Spanish speaking ones where he also works), Marchessou draws the analogy of a split personality in regard to some 
aspects of the Code, particularly copyright. That concept is very alien to many people, who are the best of people 
otherwise. For example, when some of his clients ask to make copies of cassettes, CD-ROM’s, and software, he 
explains the situation and gives the media to them instead. The concept is still a difficult one, even for people with 
decent incomes, as they view the authors as anonymous and the resources as coming from the rich north, while they 
themselves do not have the same resources. 

Ethics also applies in not influencing people to obtain contracts where influence could be wielded. There is 
an ethical divide. Some people are not at the stage where they have that type of ethical awareness, although they do 
have a strong code of ethics within their own families and on person-to-person terms. 



10 




In northern French-speaking areas, the Code is seen as a little too itemized. People take ethics for granted. 
His future translation will attempt to address these aspects. Marchessou also urges examining ways to engage in a 
dialogue about the Code with people from emerging countries. 

Multicultural “Translation” 

Annette Sherry from the University of Hawai'i, a native English speaker from the United States, is involved 
in “translating” the Code on the campus of a large multicultural university in terms of disseminating its ideas widely 
in English. She challenges her graduate students in the College of Education to develop ways to communicate the 
concepts to a multicultural audience in a very visible way. Although distribution over a Web site would be her 
students’ first choice, they are required to develop a semi permanent three-dimensional display in a prominent place 
on campus. That way they can readily reach a large segment of that educational community. 

Models related to selected principles were constructed and placed in the entry way of a classroom building 
one semester. More recently, aspects from the three sections of the Code and the complete text were produced as the 
tops of three picnic tables. Participants at the grand opening event positively affirmed that the display did inform 
them about the Code, did encourage positive attitudes and moral behavior, and were visually pleasing. Challenges 
are to have ongoing evaluation to ensure that the message is being heard and, more importantly, formally assess the 
impact of this knowledge at the schools where the pre- and in-service teachers who view the display teach. Currently, 
only incidental anecdotal information is available from the few teachers who choose to share this information. 

Spanish Translation 

Eor the Spanish translation, there were two members of the team. 

Eirst Translator. The initial translator, Sr. Diaz de Rabago from Universidad de Santiago, a native Spanish 
speaker from Spain, noted that his first draft was a literal one. After joining AECT in June 1977 and realizing that the 
association was sound, had a charter, and a Code of Ethics, he presented these aspects at a Congress in Barcelona, 
including a translation of the Code of Ethics. It was not very useful in Spain, as the translated Code was viewed as 
being very precise and legal. 

Time went on and this new request came. In a few weeks he had his first draft, but wanted second one. Some 
refer to philologists and some to the lawyers, which he sees as being worthwhile because ethics can differ from place 
to place. 

When an association, such as AECT, says it is international it needs to deal with such a problem. Sr. Diaz de 
Rabago sees a need to move in a new, more realistic direction for the Code because it is good to attack this problem 
to provide a good, clear interpretation for those in the association. It should be remembered that there must be trust 
towards the translators that are selected. 

Second Translator. R. W. Burniske from the University of Hawai'i, an English-speaker from the United 
States, approached revising Sr. Diaz de Rabago's translation as a non-native speaker. His is the perspective of one 
who learned formal Spanish in southern Spain in the early ‘80s, but later encountered a very different form of the 
language while working in Latin America. 

? Whose Spanish is this? Is it the Spanish of northern Spain, southern Spain or a Latin American 
country? 

? Who is the audience for this translation? 

? What is the context? 

In terms of whose Spanish it is, Burniske points out that the word, “shall” appears in the Code very 
frequently. In English “shall” is not “will.” In Spanish it translates as “deberan” which is “should,” “ought,” or 
“must.” “I couldn’t help wondering,” Burniske observed, “if our audience would ask why AECT has the authority to 
tell them what they should, ought, must do?” He further adds that it sounds almost biblical, as though the document 
were delivered in the form of commandments. What’s more, a term such as “harassing” reveals a North American 
bias, carrying connotations which are absent from many translations. In the Spanish language, for example, 
“harassment “would be “poco respetuosa,” which means “lack of respect,” but fails to get at the kinds of issues 
implied in the English version. 

In regard to audience, he questions whether the Code is being translated for K-12 or university 
professionals. Eor example in Section 3.1 of the Code, encourages diverse intellectual points of view. While this may 
seem a fundamental principle of Western, democratic societies and institutions of higher learning, secondary schools 
in developing nations might not readily embrace different points of views. In several Latin American countries. 
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therefore, it might be difficult for educators to remain faithful to the Code while also maintaining an amicable 
relationship within their schools and communities 

With regard to the final concern, that of “context”, Burniske cites Section 2, Principle 2.4 which discourages 
the acceptance of gifts that “might impair or appear to impair professional judgment.” At the conclusion of 
workshops he has conducted for the World Links program for educators in places as diverse as Ramallah, Uganda 
and Brazil, he has frequently received gifts from workshop participants. This presents a difficult conundrum for those 
who wish to abide by the Code: loyalty to the Code would suggest rejection of the gifts, but rejection of the gifts 
could be interpreted by local custom as a rude gesture that potentially damages the relationship that the gift was 
intended to nurture. 

Burniske urges AECT to think of the document as a work in progress, a dynamic Code. To further 
contribute to its global impact he suggests that it be posted on the Web and input be sought from around the world 
through discussion forums where readers can contribute suggested revisions of the original document as well as the 
respective translations 

Conclusion 

Although many questions remain unanswered, these early translations of the Code of Ethics will be 
available at the AECT Web site to ensure that access to the principles is offered in languages other than English. 
Other languages will be added to the Earsi, Erench, Mandarin Chinese, Portuguese, and Spanish, translations as they 
become available. The translations will be saved as PDE files to retain the unique characters of each language. 

Additional translators with expertise in both educational technology and the targeted languages are being 
identified. Exploration about ways to encourage debate about the viability of the translations within diverse cultures 
is underway. Collaboration between the members of the Professional Ethics Committee of the AECT, the translators, 
and interested members could result in the development of a Code of Ethics that is read by, and “speaks to,” all 
within the profession worldwide. 
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Virtual Reality Panoramas and Hyperlinked Digital Images: 

Tools for Qualitative Inquiry of Preservice Teachers’ Technology Learning. 

David S. McCurry 
Monmouth University 
Abstract 

This paper reports on the use of hyperlinked digital images, including still images and 360-degree 
panoramas, digitized audio recordings and online reflections by pre-service teachers used as data in 
qualitative critical inquiry about preservice teachers’ technology learning in a low socioeconomic 
district. Included are explanations of the process of image acquisition, elicitation of participant responses 
and issues related to working with this technology in analytic processes. Also discussed are the 
implications for qualitative methodology in classroom research. The latter are highlighted through issues 
related to existing development of visual ethnography and critical inquiry methods in contextual analysis 
of preservice candidates encountering populations and communities unlike their own. 

Introduction 

The classroom is a rich, cultural environment that reveals as much about the teaching process as it does 
about the cultural actors who inhabit it. One particularly significant intrusion into this environment in the past two 
decades has been the computer. Although much research has been undertaken in classrooms, understanding the 
complex realities of classroom use of technology is problematic. The characteristics of use have been linked to 
teacher attitudes and self-identity with pedagogical approaches (Becker and Riel, 2000). How teachers think about 
their own teaching and about technology are reflected in their classroom practices. Access to classrooms, for the 
researcher, is not always easy, especially in the current climate of increased security and concern for anything that 
detracts from tightly scheduled routines, adherence to structured curriculum programs, or activities not accounted for 
in school reform programs or local professional development opportunity. Even with these restrictions, many schools 
still engage in supporting observation, practicum and supervised field experiences for pre -service teacher candidates, 
typically matriculating in local college or university programs. For the university professor, these activities offer 
possibility for research collaboration, contact with local teachers and school officials, and opportunities to further the 
project of understanding education from the “inside”. 

While traditional approaches to observation in classrooms include such diverse approaches and methods as 
qualitative note taking, video and audio recording, and quantitative frequency counts of coded behavior, new 
technologies offer yet additional possibilities for data collection and analysis. Virtual Reality' (VR) panoramas, using 
available software and publishable through CD-ROM and web page development, provide a rich, contextual visual 
field which can link to document files, audio clip interview data, still images (which provide close-up, detailed image 
data of specific aspects of the learning environment), and other electronic resources. This paper describes both the 
use of this technology as a data collection and organizational method and preliminary findings from the use of this 
method in classroom research about technology. 

The materials are generated and compiled into multimedia case studies, available on CD-ROM for teacher 
candidates to browse and develop their own critical reflections. Pre -service learners are involved in discussion and 
development of the materials with local cooperating teachers, students in classrooms, and with their university 
faculty. The research into classroom practices therefore yield important study material as well as rich document and 
data sources for analysis of classroom practices related to technology use. 

The use of case studies to enhance professional development programs is well known in the fields of 
business, law and medicine but is a relatively recent innovation in teacher preparation (Lacey & Merseth, 1993). Case 
studies allow preservice and in-service professionals opportunities to explore critical issues through real examples, 
drawing on familiar contexts and situations, with an expected high degree of transfer to future practice. Traditional 
case studies are print text oriented, sometimes with supporting photographic, audio or video recording 
documentation. These are typically offered in a parallel construction where participants follow-along with the text. In 
these latter examples, cases are typically descriptive in nature, offering complete pictures of a specific scenario. 
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actor/participants, assumed problems, which are pre -defined, or assumed to be obvious, and directed, expectant and 
narrowly defined outcomes. Multimedia technologies, especially digital images, video, audio and electronic text, offer 
the possibility of non-linear, hyperlinked constructions which allow learner-defined and directed as opposed to 
‘scripted’ access to and organization of the information. Outcomes of scenarios, which are rich in detail and context, 
can be multiple and need not be pre -defined within the material itself 

Another aspect of the methodology of using cases in teaching is in the promotion of “case-based 
reasoning,’’ where analysis of documented real, or near-real situations or scenarios lead to problem-posing, or 
“problematizing’’ (as a construct of critical pedagogical practice), hypothesizing, critically contrasting possible 
solutions, and problem-solving. Problems, in these contexts, are typically of the “ill-structured’’ variety (Simon, 1973, 
1984; Voss & Post, 1988; Jonassen, 1997), where many variables may interact and multiple possible outcomes or 
solutions may present themselves. The application of problem solving structures and research has also been 
extended to interactive hypermedia scenarios (Gibson & Albion, 1997) and has drawn great interest in educational 
research that examines the cognitive foundations of learning and classroom practice. 

Multimedia case studies also provide rich, complex, and contextually authentic examples of classroom 
practice and information for analysis, discussion and learning (Goldman, Barron & Witherspoon, 1991; Merseth & 
Lacey, 1993). Case examples are often used as discussion material to help candidates develop constructions of “right 
and wrong’’ approaches related to various methods and theory. Video case studies in particular are often constructed 
to represent “best practice’’ by simulating synthetically constructed scenarios (both good and bad) for the 
participant to interact with and to choose from a limited range of available outcomes. Less common, however, are the 
uses of case examples to facilitate critical inquiry, as most case studies are either fictional, focus on divining 
“appropriate’’ responses to fixed scenarios, or are otherwise removed from more immediate contexts of the learner. 
Reflexive, critical discussion between the new preservice teacher and the more experienced host teacher is often 
curtailed in typical field experience settings owing to time constraints, miscommunication and the relative inequity of 
university student and professional teacher relationships. Rather than exposing candidates to a collection of “right 
and wrong’’ exemplary cases on videotape, the use of a critical approach engages local teachers and teacher 
candidates in the self-critique and narration of their own practice. In this way, teachers are highly involved in the 
development of the material along with university faculty and preservice certification candidates. 

The practices involved with collecting or creating scenarios for cases to be included in various case 
methods approaches are also varied. These can range from complete works of fiction (grounded in familiar realities 
and experiences) to cases representing actual events in classrooms and in schools. In the latter examples, the 
classroom events are documented through any one or a combination of video, photography (digital and analog), 
sound recordings, documents and narratives derived from observations, self-reflections and interviews. In selecting 
the classrooms to study, the researcher purposely did not position “best example’’ teachers or classrooms in the data 
collection process. The idea was to collect more of an authentic glimpse into classrooms and school environments 
which would further be open to analysis and observation by pre -service teacher candidates as well as the researcher. 

Purpose of the study 

This study is both an exploration of new methods of qualitative research using multimedia technology, and 
an inquiry into how teachers utilize technology resources in the classroom. The presence of technology, and the 
expectations for its use, are factors shown to be related to teacher’s pedagogical method and self-efficacy (Becker 
and Riel, 2000). Through the use of these qualitative tools, a better understanding of classroom practice with 
technology might be revealed in teacher’s own reflections on the technology as an artifact in the unique environment 
of the classroom. This research is directed at finding out two things; 1) What are teachers doing with technology in 
their classrooms and how do they think and discuss these practices when reflecting on and in their own classrooms, 
and 2) How can VR panoramic images be used in such qualitative inquiry to elicit teacher meaning (about 
technology) and provide better understanding about the relationship between pedagogical practice, technological 
interventions, and the school culture and environment? 

Methods 

The methods used in this inquiry are informed by ethnographic traditions, particularly visual ethnography 
(Collier & Collier, 1986; Collier, 1995; Pink, 2001) and methods used more typically in visual sociological studies 
(Banks & Morphy, 1997). Such methods seek to uncover and reveal patterns in meanings and definitions as 
represented in description and analysis of documents, interview transcripts, and elicited comments from visual 
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stimuli, such as photographs and video or film and are broadly referred to as “image-based research” (Prosser, 1998). 
Recently, the availability of VR images (rotational objects, panoramic images), provide additional materials to use as 
data collection tools in interpretive inquiry. In particular, photographic images used as materials to elicit comments 
for analysis purposes have been used in teacher professional development (Kanstrup, 2002). 

In collecting images and interview recordings for developing multimedia case materials, it became obvious 
that a rich source of data for inquiry into classroom practice, along many dimensions, were being accumulated. The 
computer software used to create the panorama images, hyperlinked environments, and output to multimedia content 
was The VR WorxTM by VR Toolbox. The software is available for both Macintosh and Windows platforms and can 
be used to create panoramas, rotational objects, and “VR scenes” where multiple panoramas are linked together. The 
implications of this feature are discussed below. 

Qualitative software (QSR NVivo™) was used to encode participants’ responses from audio and video 
interviews, university student online discussion postings, and other relevant documents. Such computer assisted 
encoding and analysis is becoming well established in qualitative methods (Kelle, 2000). Encoded texts were linked to 
visual nodes in the materials to establish data triangulation. Emergent discussion reveals (as an ongoing study) 
patterns in the self-efficacy of preservice teachers, working with rich multimedia technology and materials. Teacher’s 
ability to relate the use of technology in their classrooms is related to their self-perceptions of the learning 
environments they create. In the particular context of the field study in which these case studies are drawn, the 
socioeconomic status of the district (low, working class) contributed to the motivation for innovation and school 
renewal. The latter is a response to external pressures from state and national government movements (court 
decisions and state educational policy), and an historical identity in the community, which positions the community 
as an “underdog” (underclass) community in the local region. The reflexive documentation of the school community 
by university students, classroom teachers, and university faculty provides a unique example of multimedia 
technology use in a qualitative study. Implications for collaborative study and research, using the qualitative 
software will be discussed. 



Findings 

The teacher interviews relate particular attitudes, anecdotes, and reveal value assumptions about 
technology and their own practice. The VR panoramas provide a very powerful way of letting the teacher manipulate 
the images and create a process of “self-elicitation” where they talk, during the interview process, about their own 
classroom practices and specific instances of using the technology. Such results are rich in detail, many more times 
than would be collected from surveys or structured interview results. One teacher reveals, for example, when viewing 
the classroom panoramic image, that the computers had been given to the classroom as part of the whole school 
reform program. The indications in her coded text reveal further assumptions about the non-investment which 
teachers feel when the technology is given from “the top down” (i.e. through administrative directive). Problems 
encountered with technology (“That new Dell over there, for some reason. . . which never seems to work”) are also 
pointed out in the process of teachers reflecting on these VR models of their own classrooms. 

The other data set, pre -service teacher candidate reflections provided through their online course support 
area, gives additional reason to believe that teachers do not utilize technology in accordance with emerging practices. 
Their observations, converted to text and coded in the qualitative software, are matched to nodes in the classroom 
teachers’ elicited comments and reflections on their own technology use. Specific instances or events, with shared 
meanings, are open in these data constructions to intersubjective analysis. What teachers say about their own 
technology use may, or may not, relate to observations made by their pre -service teacher students. The process of 
reflexive dialogue, created through the use of the materials, holds potential for further developing the case analysis 
as a method of both pre -service teacher and professional teacher development. 

Creating the multimedia materials is time-consuming and requires a relatively high level of technological skill 
with specific software routines. Eamiliarity with multimedia technologies is highly recommended. The cost of the 
software and the camera equipment (tripod, recording devices, etc.) was not exceedingly high. 

Discussion and methodological implications 

The technology described in this study has been widely used in developing “virtual tours” for the real 
estate and tourism industries through online marketing programs, and in product marketing and promotion where 
products (as objects) can be examined in virtual, 3 dimensional rotations. The use of this technology in qualitative 
study represents another advancement in building interpretive contexts that are founded on visual sociology and 
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anthropology. As an exploration of new methods, the virtual images of the classroom created with the teacher’ 
involvement become part of the process of eliciting meaning about technology practices and classroom culture. The 
resulting data in the form of hyperlinked nodal points within the images offer levels of interactivity and opportunities 
for exploring which far exceed static images or viewing analog videotapes of classroom practice. While the latter can 
be coded and analyzed using any one of many existing protocols, the “frame” of the image is fixed during recording. 
VR panoramas provide a moveable frame, giving control over the selection of objects and detail to the viewer. This is 
a departure from previous visual data collection and analysis tools. 

New technologies usually bring one of several new characteristics to existing practice. One distinct 
characteristic of the panoramic material is in the ability to create a near-sensation of standing in the physical 
environment of a classroom, school hallway, playground, or one of the many other spaces of schools as cultural 
domains. Unlike static, still images - though these can be very rich, highly discrete forms of visual data - or video and 
film motion images, the visual panorama provides the ability for the viewer to control the frame of visual reference, 
scanning back and forth, up and down, while selecting specific points of interest in the visual field to locate and 
consider for analysis. Further, because of the digital ability to hyperlink the image to external sources (images, video, 
audio clips, or other panoramas), the possibility of creating virtual interactive maps of school cultural environments 
exist. 

The “Scene” capability of the VR Worx software, in creating multiple, linked panoramic images holds great 
potential for building rich visual contexts for interpretive analysis. Such scenes can be joined to form a large-scale, 
visually documented and navigable “virtual” reality for elicitation of participant comments, analytic critique, and 
analysis within a variety of applied frameworks, such as program evaluation, cultural and communicative practices 
within the school environment, and more pragmatic concerns of pedagogical practice and efficacy. 

In one potential application, internal scenes of classrooms can be linked to hallways, which are in turn linked 
to school exteriors, which are linked to further locations in actual community environments. In this way, an observer 
accessing this material could literally navigate his or her way from a point in a classroom, through the door leading 
into the hallway, down the school building corridors, perhaps stopping in the office to interact with recorded 
interviews with school administration, move out into the entrance, through the doors, and out onto the street. 

The data, once collected and analyzed, can also be re -constructed into the ethnographic case-example for 
further exploration by other pre-service learners or researchers. The digital format of the materials is also potentially 
well suited for collaborative and distributed qualitative study. While standardization of qualitative practices is a 
problematic area in methodology, online collaboration is being facilitated through various projects (see Kanstrup) 
and could be used to form large-scale connected ethnographies of classroom studies with rich data sets, and 
intersubjective interpretation across large numbers of participants. 
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‘ Although the term “virtual reality’’ is commonly used, precise definitions do not yet exist. In this study, the 
panoramic images are termed “virtual reality” by the developers and vendors of the software used to create and 
transmit the images and the use of the term in this paper is consistent with that definition. 
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Evaluations of reflective practice: The design, development, and implementation of 
an advanced graduate course in Web-based instructional design 
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Abstract 

This paper discusses the design, implementation, and adaptations of a graduate-level course in Web- 
based instructional design and development. Emphasis was placed on learners’ self-regulation and 
reflective practice as they explored the application of instructional design and development concepts and 
principles. These included pedagogical, technological, organizational, and ethical issues related to the 
design, development and assessment of Web-based instructional design. 

Introduction 

Design and Development of Web-based Instruction (WBI) is an advanced instructional design course. 
Employing the apprenticeship model introduced by Donald Schon (1987) to encourage reflective practice and later 
within the context of instructional design courses by Ertmer and Cennamo (1995), students engage in authentic 
design activities in a studio environment. 

The intent of WBI is to equip students with a real-world experience that will help them transfer what they 
learn in graduate school to professional practice. In this case, the learning environment integrates classroom (face-to- 
face) and virtual (Web-based) environments. The unique skills and life experiences of each class member serve as the 
foundation for collaboration and productivity. 

The objective of this course is the application of instructional design and development concepts and 
principles. These include pedagogical, technological, organizational, and ethical issues related to the design, 
development and assessment of Web-based instructional materials. Learners apply relevant instructional design 
principles that include the analysis of complex instructional design situation for a client and designing, developing 
and evaluating a Web-based intervention strategy. The students develop their interface design skills through the 
evaluation of Web-based instructional and resource materials and base the design of their instructional Web site 
functionality and look and feel on usability guidelines. 

Background 



The Reflective Practitioner 

Preparation for professional practice, in any design profession, presents unique challenges to academic 
programs. While students of design readily grasp the technical processes and theoretical foundations that serve as 
the framework for their professional practice, many struggle to transfer what they learn to the instructional design 
workplace. As novice designers, they often adopt a mechanistic approach to the models that guide practice because 
they lack the experience that will enable them to reflect on and evaluate the multiple factors influencing a situation. 
The problem solving and generation of solutions in complex situations, which require components of reflective 
practice such as intuitive, creative and critical thinking, challenge the design students and their instructors (Schon, 
1987). Reflective practice refers to the metacognitive processes, the internal and external conversations, that experts 
engage to improve practice by drawing connections between theory and practice while making judgements about 
ideas and actions (Hinett & Weeden, 2000). 

Problems that mirror the complexities and issues that exist in real-world problems can promote reflective 
conversation among novices when accompanied with a variety of examples early in the learning experience and 
encouragement to share their perceptions on the process. Learners practice solving problems as they become aware 
of the metacognitive strategies that experts use identify and analyze complex situations (Gage and Berliner, 1984). In 
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doing so, the learners explore relationships between cause and effect, the interdependence of the component parts of 
a design process, and begin to recognize the significance of the rules, theories and procedures that form the 
foundation for practice. The learners begin to internalize the design process. 

Schon (1987) suggested using a practicum studio experience that promotes reflection under the guidance of 
an expert practitioner who coaches the students as they solve problems and generate solutions. He described the 
practicum studio as a virtual world, designed to approximate real-world situations, while providing novices a means 
to develop their skills without the consequences of making errors in an actual work environment. The expert- 
instructor's modeling of the thought process helps learners develop an awareness of quality and of the 
interdependence of the component parts of a systems. They begin to recognize the significance of the rules, theories 
and procedures that form the foundation for reflective practice. Schon felt that this provides a means for learners to 
internalize the design process. 

Reflective ID Practice 

Over the past decade, numerous instructional strategies, including Schon's practicum studio, have been 
implemented in graduate ID courses in an effort to increase the breadth and depth of the ID experience that novices 
can take with them into the workplace. Many of the strategies, adopted from other design fields, such as architecture 
and engineering, successfully implemented open-ended learning experiences that encouraged reflective practice. 
Their objective was to incorporate some of the skills that research into professional practice and cognitive learning 
theory uncovered such as collaboration, problem-solving, management, and effective, yet flexible implementation of 
the ID process. Other strategies for promoting reflective practice included methods such as real-world projects, 
apprenticeships, and case methods (Rowland, Parra and Basnet, 1994). 

Studies into the effectiveness of these learning environments demonstrated that students were enthusiastic 
about the opportunity to practice their skills in novel situations under the guidance of expert instructors (Ertmer & 
Cennamo, 1995; Julian, Kinzie & Larsen, 2001). Ertmer and Cennamo (1995) also explored the use of real-world 
projects, but focused the students on their cognitive and metacognitive processes rather than simply on procedural 
skills engaged during the design and development process. The 'expert' instructors modeled their cognitive and 
metacognitive skills by talking aloud as they reflected on design issues and provided the students with feedback on 
their projects. Ertmer and Cennamo found that this was a positive learning experience for their students and observed 
them taking the time to think through the process before jumping to the generation of solutions. They felt this 
resulted in an increased understanding of quality design. While some students excelled in these open-ended 
activities, however, others struggled without a concretely -defined problem or failed to draw connections between the 
foundations they had learned and the process of solving ID problems in a complex situation (Julian, Kinzie & Larsen, 
2001). Ertmer and Cennamo (1995) also found that the students required coaching throughout the process to keep 
them from getting too far off base and they benefited from frequent encouragement to consider multiple perspectives 
and reflect on their decisions. We have found that the instructional design of on-line materials increases this 
challenge, adding interface issues such as functionality, accessibility and look-and-feel as well as programming, 
monitoring and site-maintenance. 

Our advanced instructional design course. Design and Development of Web-based Instruction (WBI), seeks 
to meet this challenge with strategies such as expert modeling and peer review that encourage behaviors 
characteristic of the reflective practice exhibited by expert designers. The students engage in authentic activities and 
develop products for actual clients as they consider the pedagogical, ethical, technical and organizational issues 
related to Web-based environments. 

In this paper, we discuss the design and implementation of Design and Development of Web-based 
Instruction. We describe the adaptations we made prior to the second offering of this course to enhance the 
students' development of their reflective practice skills. This included the addition of practice on-line evaluation 
activities that introduced the students to self-regulation and reflective practice before they began their ID projects. 
When used in conjunction with objective evaluation criteria, we found that this provided the students with an 
experiential foundation that enabled them to organize and communicate their ideas as they designed and developed 
Web-based instructional modules. We also describe the evolution of the students' perceptions on evaluation, peer 
review and the instructional design process. While all of the students embraced the benefits of peer review and 
feedback, some continued to struggle with the cycle of feedback and revision, especially during the design phase of 
their projects. 

WBI: Semester One 
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To facilitate reflective practice, we encouraged our students to apply their instructional design skills in a 
variety of situations. Teams of two to four members completed a real-world project for an external client, and 
participated in problem-solving activities based on project excerpts and case scenarios. Their process of reflective 
practice involved discussions and presentations that solicited facilitator and peer review and feedback. As 
facilitators, we used the think aloud process to model our analyses and reflections and the students were encouraged 
to provide each other with constructive feedback that would enable their peers to improve the efficacy of their 
designs. We introduced the students to the concept of reflective practice and invited them to read about it on our 
class Web site. 

Authentic Design Projects 

The students began the semester with the task of identifying a client with a problem-based instructional 
need that might be resolved using a Web-based environment. The students presented their project proposals, and 
the instructor facilitated selection of projects and teams based on the type of need and the feasibility of completing 
the project in one semester. The student teams then conducted a needs assessment in which they used the results of 
an environmental analysis to specify the need and describe the characteristics of the target audience, stakeholders 
and the practice and work environments. The project teams followed a timeline that integrated the generic 
instructional design model with prototyping and usability practices (Table 1). They presented their progress to the 
class and participated in review and feedback sessions with one or two teams each week. Following completion of the 
major components of each stage of the design, the teams also met with their clients to review the design and discuss 
revisions. 



Instructional Design Project Timeline 


Week 2 


Idea generation: Identify target audience prototypers and formative evaluators 
: Client/assumed need 


Week 3 


Instructional Design Needs Assessment 


Project Management: Design Schedule, Time Logs 


Week 4 


Goal Analysis/Task Analysis 


Weeks 


Objectives and Assessment Items 


Week 6 


Instructional Strategies: Development Needs Assessment 


Instructional Strategies : Selection 


Content Analysis: Content Map 


Week? 


Functionality Design 


Week 8-10 


Prototype I 


Revisions/Prototype II: target audience, client, target audience instructor (if applicable), 
content expert 


Week 11 


Web Development/Project Management Development Schedule/Time Logs 


Week 12 


Formative Evaluation 


Week 13 


Revisions 


Week 14 


Final Submission/Final Presentations 



Supplemental Activities 

Although basic technology skills were prerequisite to this course, the students explored the framework and 
logic of Web development technologies. They completed mini-projects such as Photoshop collages, button and 
background graphics, and development of pages scripted in HTML. Discussions followed, and the students were 
encouraged to reflect on their design and Web-user experiences as they discussed the literature about current 
technologies and functionality design. Case scenarios, which were used to model new concepts and discuss complex 
situations, were discussed in the course on-line discussion area. 

Initiating Reflective Practice 

When we first implemented this course, we found that most of the students were enthusiastic about the 
studio environment. They saw this as an opportunity to design products in an environment where they could benefit 
from the guidance of an expert designer. The students exhibited reflective practice to varying degrees over the course 
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of the semester and we found that it was most evident when they engaged in general discussions or the analysis of 
scenarios. We observed that they hesitated to engage this process, however, when they met to discuss and present 
their projects and related activities. They often struggled with their role as peer-reviewers and were apprehensive 
about offering feedback that did not praise their peers' work. Comments tended to focus on compliments about effort. 
The students also struggled to explain their rationale for design decisions and, when students did offer constructive 
feedback, some of the recipients were reluctant to consider the opinions of others. 

Through facilitator coaching and modeling, however, the students' hesitancy to engage in reflective 
discourse began to dissipate as assignments neared completion. —By the end of the semester, several of our students 
engaged in discussions that focused on the tasks at hand and described their rationale for decisions and potential 
solutions. On an informal basis, some of our students shared their perspectives about their initial reluctance to 
discuss their reflections on their own projects or provide feedback to their peers. They noted that it had taken 
considerable time for them to regard themselves qualified to evaluate their decisions or those of their peers because 
this had been a new process for them and a new way of thinking about design and development. W e determined that 
limited experience in this role, especially in the evaluation of on-line materials and, consequently limited confidence, 
created a barrier to effective participation in this process. —Just as designers draw upon prior experiences when 
analyzing problems and identifying solutions, novice designers require experiences that they can draw upon when 
engaging the metacognitive skills that define reflective practice. 

A Model for Reflective Practice 

We decided to adapt our course and build a model for reflective practice by: 

• Introducing the students to the process of reflective practice before they were called upon to evaluate 
their designs or those of their peers, and 

• Equipping them with objective tools for critical review and feedback so that they felt qualified and 
objective in their decisions. 

The MERLOT Portal (www.merlot.org) is a collection of on-line instructional materials, expert and peer 
evaluations, and criteria for reviewing on-line instruction. Noting that these components could mirror our class 
project review and feedback system, we chose the MERLOT Portal as the springboard for our students' introduction 
to reflective discourse, and designed an experiential activity. 

Activity Guidelines 

The activity began with student creation of evaluation templates that were based upon Merlot’s on-line 
instruction criteria and the instructor's interface design guidelines. The templates then served as guidelines for 
individual, team and whole group evaluations of instructional materials linked to the Merlot portal. The students then 
compared their evaluations to Merlot’s expert and peer evaluations. Eollowing each stage of this activity, the 
students participated in on-line discussions that were facilitated with questions about the evaluations and reflective 
practice. The first discussion focused on template design and the students' evaluation experiences and later evolved 
into a discussion focused on the interrelationships between reflective practice, evaluation, expert and peer feedback. 
In the final discussion, the students shared perspectives on transferring this experience to their project design and 
development. 

Our first participants in this activity were nine students who were enrolled in Web-based Instructional 
Design and Development, an advanced ID course. The students worked in instructional design teams that had been 
formed for their semester projects. Their introduction to the activity described a process for critically evaluating on- 
line instruction so that they would be equipped to evaluate their own designs and offer feedback to their classmates. 
The students used the following guidelines (Table 2) to complete their activity evaluations. They presented their 
findings in class, compared them to the expert and peer reviews on the MERLOT site, and discussed the similarities 
and differences between the evaluations. The activity concluded with the final on-line discussion about transfer 
(Table 3): 

Table 2: Site Evaluation 



A. MERLOT Site Selection: www.merlot.org 

1. Erom the subject links, select the content area related to your ID project topic. 

2. Eollow the sub-topics and identify two or three interactive Web-based instructional sites. 
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i) Do not select informational sites 

3. Do not view the peer or expert reviews (We will explore these reviews at a later date.) 

B. Template, Review and Evaluate 

1. Download two documents for this activity: MERLOT Evaluation Criteria and Usability Guidelines 

2. Work with your team to create evaluation templates based on the MERLOT Evaluation Criteria and 
Usability Guidelines. 

i) Leave spaces so that you can fill in the templates when you evaluate the sites you identified. 

3. Have each team member evaluate one or more of the sites 

i) Eor criteria that does not apply to your site, mark, ”N/A" 

4. Exchange evaluations with your team members. 

i) Review the site(s) and add comments. 

ii) Second reviewer: you may want to use a different color ink so that you can remember what you 
added 

iii) Discuss your results with your team’s members and create a final copy of your evaluation for 
the class. 

C. Strengths and Weaknesses 

1. Create a list of the primary strengths and weaknesses for each instructional site your team reviewed. 

2. Create a discussion thread (linked to this discussion forum) for each site and enter the name in the 
subject line 

3. Start your message with the site's address and follow with your list of the site's strengths and 
weaknesses 

4. Prior to our next meeting, return to the discussion forum and review the evaluations. 

5. Post a comment sharing your perspective on effective Web-based ID. 

6. Prepare a brief five-minute presentation on the strengths and weaknesses of the Web sites you 
evaluated. 



Ta ble 3: Reflective Practice 

Now that you have completed the MERLOT activity, evaluating content, design, usability, and comparing 
your results to the MERLOT evaluators, the next step is to consider your design and peer review of our class 
projects. 

Please complete the following: 

A. Literature summary on reflective practice 

1. Review the three pages of literature on reflective practice in the "About" section of our course Web 
site 

2. Consider what it means to: 

i) Reflect before you design (reflection before action), 

ii) During design (knowing and reflecting during action), and 
hi) When reviewing your project or peer projects (on action) 

B. Post comments to the following discussion threads 

1. How can your experience from the MERLOT evaluation activity transfer to self-reflection (Before, 
During and On -reflection) as you design and develop your project this semester? 

2. How can your experiences from the MERLOT evaluation activity transfer to your critical peer 
feedback that you provide to the other teams about their projects? 

How can your experience evaluating and reviewing critiques through the MERLOT activity help you process 
and apply feedback from peers when revising your ID project? 

What did we learn from the MERLOT Experiential Activity? 
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Modified templates form objective evaluation tools 

The production of templates required negotiation among team members and careful review of the evaluation 
tools they were adapting. In addition to negotiating the guidelines we provided, most of the participants explored the 
design Web sites we recommended, the most popular being www.usability.gov. All of the participants felt that this 
process helped them to explore and understand each criteria item. As one student noted, customizing the templates 
helped her team to become "so familiar with each concept” that they were able to use their template to objectively 
review a site and identify "not only what worked, but ways to improve the learning experience." 

Another found that the template creation focused her evaluation. 

I think one of the most important things that I learned from the MERLOT evaluation was from 
creating the templates. It forced me to read and extract the most important points from the 
guidelines and use those points to evaluate different sites that were totally different. Personally, it 
helped me focus on important aspects of the site without wasting time on superficial things. The 
result was a well-oriented and rich evaluation. 

Expert evaluation comparisons confirm qualifications 

The class discussion on Merlot’s expert and peer reviews focused on the subjective nature of evaluation 
and the students' recognition that they were qualified to share their perspectives on effective design. This was 
evident in the on-line discussion about important characteristics of effective Web ID. In addition to listing 
instructional and interface characteristics of effective on-line instructional design, seven of the nine participants 
noted characteristics that are commonly overlooked when sites are designed or evaluated. 

I think it is important and often overlooked that the content is appropriate for the medium. If 
objectives state a learner will be able to dp_ something, then it must be testable via the medium. 

Otherwise, a blended solution should be offered that allows assessment as an off-line activity. 

We used complex criteria to evaluate the site, but it shows how the type of evaluation criteria can 
affect rating results. 

We think the MERLOT evaluator could have focused on a variety of criteria that are important from 
a teacher's point of view (see our attempt to categorize those criteria in our evaluation templates). 

Six of the participants also noted that the comparison provided ways they could "add to their review 
guides” and improve their evaluation tools. 

Self-reflection means self-awareness about the development of expertise 

The participants felt that the MERLOT activity was a springboard for their development as Web-based 
instructional designers. They shared comments on ways that it advanced their ability to engage reflective practice. 

The MERLOT evaluation activity allowed me to reflect on my own actions; it challenged my view on 
characteristics of Web-based instruction... lam a little bit closer now to my goal to become a 
reflective practitioner who is not afraid to experiment or change. 

The activity has helped me frame web site design into three general areas: content, potential as a 
teaching/learning tool, and usability. This gives me a framework for dealing with web design in 
chunks instead of one overwhelming creation. These areas help me focus my solution. 

The students expressed in class discussion and again on-line that the "MERLOT activity provided structure 
for the review and feedback process,” 

The MERLOT activity gave us a vocabulary that we accept and experiences to draw upon. 
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The MERLOT activity will help me to build a mental map of my site in my head... an internal library 
and a talking reference tool we can draw upon. It is important to call on this information as we 
evaluate obstacles and identify solutions in our own projects. 



Peer review is a conduit to effective communication 

The participants focused on methods for communicating constructive feedback without insulting their 
peers. Again, noting that the template enabled them to be objective, they shared ideas for encouraging their peers by 
"discussing problems as opportunities for improvement." 

Our templates and the MERLOT site gave us a language and structure for the exchange of ideas, 
which is likely to make feedback more constructive and objective. With detailed specific feedback, 
those who receive criticism have of better chance of incorporating improvements. 

Having guidelines will help to be more thorough in peer reviews because we have specific things to 
look for rather than offering vague "looks good" types of feedback. 

The majority of the participants also noted that they now realize that peer review is a part of regular practice. 

Some of us may be reticent and we may not want to critique others, but knowing this is an accepted 
professional activity makes it easier. 

Transfer to Instructional Design Practice 

When the students began producing drafts of their work on their ID projects, we directed them to mirror the 
MERLOT activity by posting their projects on-line and soliciting peer reviews. Only one team posted a draft and one 
participant offered comments. The teams brought drafts of their projects to our next class meeting, however, ready 
and eager to solicit feedback from their peers. They explained that they had not felt comfortable posting drafts on- 
line. They wanted the opportunity to share their reflections on their designs with those reviewing their work. The 
face-to-face peer review of the ID project drafts was productive; the teams helped each other identify critical 
revisions that would improve their designs. They accompanied each comment with a rationale for their perspective or 
decision. 

In our follow-up discussion, the students discussed why they felt that peer review of their own projects 
should be facilitated in face-to-face meetings. The students felt that their evaluation templates gave them a common 
and reliable vocabulary for peer review, but they wanted to be able to accompany their opinions with an explanation 
for their decisions. They also explained that it was easier to discuss practice activities such as the MERLOT 
evaluation on-line because there were no consequences if their recommendations were inappropriate. They felt that 
reflection on action when reviewing actual ID projects should involve face-to-face discussion where designers and 
reviewers could evaluate and solve problems together. On a personal level, they felt uncomfortable putting drafts on- 
line because, "we are still learning how to design" and they noted that the instructor played a significant role as a 
reflective practitioner during project discussions. This informed us that we needed to help the students transfer the 
reflective practice they engaged during the MERLOT activity to their ID projects by allowing the students more time 
to review each others' work in face-to-face sessions. 

Student participation in on-line peer review increased, especially when the students approached project 
deadlines and posted requests for assistance. In addition to critical reviews of projects, some of the class members 
posted references to Web sites and articles that they thought would help their peers on other projects. 

Grades and Reflective Practice 

When the students began to prototype and develop their instructional Web sites, both face-to-face and on- 
line venues were populated with drafts, links to sample Web pages, calls for feedback, and recommendations. As the 
students moved into the final phase of production, however, the students became concerned with the encroaching 
deadline and the instructor's final evaluation of their modules, which would result in their grade for the semester. 

They began to adjust their designs to what they "thought the instructor wanted" and called upon the instructor more 
frequently for revision guidelines. We met with each team to encourage the students to draw upon their own 
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expertise as well as that of their peers. As one student expressed, the feedback he received from each of the students 
and the instructor presented such a wide range of recommendations that he had been uncertain "who to listen to.” 

Next Steps 



A reflective model for practice 

As a first step to reflective practice, the novice designers gained experience reviewing instructional materials 
and making recommendations. The students were able to recognize the value of their own prior knowledge through 
practice and discussion activities. And finally, the evaluation guideline templates provided them with a tool that 
served as a structured format for evaluation and a common vocabulary for peer review and discussion. 

Refining Reflective Practice 

Although we assured the students that the team projects should reflect their knowledge of instructional and 
interface design and consideration of peer critiques, the students did not trust this new process enough to carry it 
through to the final evaluation and their grade. We resolved this problem by using coaching questions to talk the 
teams through their final revisions, eliciting their ideas and working with them to evaluate and select those that would 
enhance the effectiveness of their design. We determined that we would need to identify a new grading scheme that 
more closely aligned our model of reflective practice with the criteria we used to evaluate the final project 
submissions. 

Institutionalizing Reflective Practice 

Further research into systemic use of reflective practice in graduate programs can help us determine its 
influence on problem-solving and the transfer of these skills to the workplace. We have begun to extend the use of 
our reflective practice model in two of the core courses in our educational technology program. We are finding that 
each environment demands different methods for the introduction and implementation of these skills and we continue 
to research ways to adapt this model for use in the applied courses throughout our program. 
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Wired and Wireless Learning Environments at Minnesota State University/Mankato 

Tim Kochery 

Minnesota State University/Mankato 
Abstract 

Recently, some major technology integration projects have been completed at Minnesota State 
University/Mankato. These projects involve incorporating classroom multimedia equipment into many 
classrooms, as well as pursuing mobile learning experiences through wireless connectivity. MSU is a 
four-year comprehensive university that serves 13,000 students and offers more than 150 majors and 71 
graduate programs. Over the past two years, MSU has progressed from having very few classrooms 
outfitted with contemporary A/V equipment, to the installation of state of the art presentation systems in 
every General Education classroom on the campus (102 rooms). Simultaneously, MSU has evolved from 
an institution without wireless connectivity, to a campus where faculty and students can be observed 
using portable computing devices to establish network connections anywhere, thus providing un-tethered 
learning opportunities. In a recent survey of 1,300 institutions across the nation, MSU ranked in the top 
100 for wireless access and networking ( www.wiredcolleees.com) . Currently, we are engaged in faculty- 
sponsored projects that are investigating effective strategies for implementing mobile, hand-held wireless 
tools (Compaq/iPAQs) into a variety of instructional applications. Furthermore, we are working with 
our course management system provider, Ucompass/Educator to develop course content, articles and e- 
books for the small screen design, as well as expanding the capacity for portable wireless 
communications. These wired and wireless technology advancements have greatly influenced the 
instructional capacities of the faculty and afforded new learning environments for the students on the 
MSU campus. 

Context and History 

Over the past 5-10 years, MSU has been faced with many concerns regarding the purchase, implementation, 
and integration of electronic communications technologies that will best serve the academic goals of this institution. 
These challenging decisions are similar to the difficulties faced by many schools that are presently faced with 
declining budgets, while at the same time trying to respond to escalating demands from students for access to 
computer and communications technologies. For any educational organization, this situation causes much 
uncertainty, which leads to the need for a re-evaluation and possible re-alignment of the goals and services that can 
be maintained. MSU encountered these same obstacles and was also plagued by concerns about the costs of these 
technologies, plus how these tools and applications would continue to transform. In addition, many people 
questioned the validity of making considerable investments in technology, and advocated that these expenditures 
would have insignificant impacts on the teaching and learning environment. This created a hesitant campus culture 
towards technology and very cautious advancements had been made regarding hardware and software purchases. 
Furthermore, because there was a lack of campus consensus and commitment, many Departments engaged in a 
variety of uncoordinated and independent initiatives. These projects and implementations achieved limited success, 
because they lacked centralized support and suffered from dubious objectives and vision. 

These past failures and expenses caused MSU to re-examine its vision of the educational services that 
would be the primary objectives of this institution. It was agreed that the majority of the students that we served 
were residential or local students who attended classes at our physical campus. Therefore, although expanded 
distance learning programs are certainly a fundamental aspect of our future plans, our primary needs involved 
improvements to the technology capacities of our existing bricks and mortar environment. A needs assessment was 
conducted to identify the technology constraints and resources that existed at MSU. Many interviews and 
observations were performed, as weU as open-meetings with both faculty and students were conducted. The 
conclusion of this evaluation process was a campus-wide survey, which helped create the critical technology 
objectives that matched to the educational missions of this institution. These objectives became the foundation for 
establishing central goals and a meaningful technology plan for the entire campus. One of the primary goals that 
emerged was the necessity of improving the computer technologies available in the existing classrooms. This was 
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considered a priority because these enhancements would enable the integration of web-based and multimedia 
resources into course instruction. 

Upgrades to Wired Classroom Environments 

As of January 2000, only 3 classrooms existed with built-in computers, and/or A/V equipment configured 
with a permanently installed video/data projector. Mobile carts outfitted with presentation equipment were available, 
but these needed to be reserved in advance and delivered to the appropriate classroom. In essence, this dependence 
on reserving multi-media carts penalized early faculty adapters, because they needed to remember to make these 
scheduling arrangements, plus rely on that equipment arriving on time and in good working order. Furthermore, while 
training sessions on PowerPoint, Adobe Photoshop, Authorware and other software programs were provided, there 
were no readily available facilities to incorporate these applications into classroom instruction. This created severe 
barriers towards faculty interest in learning about these technologies or using them as a delivery tool. 

It became very apparent that a critical need was to make emerging computer presentation equipment 
ubiquitous and accessible throughout the university. Our goal was to make this multimedia equipment as prevalent 
and seamless a "tool option”, as the whiteboards that instructors have used for many years in the classrooms. 
Open-campus meetings were conducted to obtain both student and faculty feedback on what technology should be 
made available. This input and participation was critical as these were the clients who would be most affected by 
these implementations. We began our installation process by focusing on twenty of the larger classrooms that are 
most heavily used, and introduced the concept of "classroom tiers” to identify the type of equipment that would exist 
in certain rooms depending on their size and media requirements. 

In the summer of 2000, we began the installation process and created 6 Level III classrooms and 14 Level II 
classrooms. Every equipment configuration is comprised of an instructor control station, a media controller/switcher, 
a ceiling-mounted projector plus screen, and an audio system for media playback. Included in each control station 
are video, audio and power cables for a laptop computer interface and Internet access. The Level III classrooms are 
upper-end rooms with a more complete set of equipment and controls including built-in computers (both Mac and PC 
format), as well as the plug-ins for interfacing with a personal Laptop computer. Another feature of these rooms is 
that they contain both a console mounted microphone, as well as a wireless remote microphone for amplifying the 
instmctor’s voice. These Level III rooms have a full-size, color, "touch screen” for controlling and switching between 
the presentation equipment: the ceiling-mounted video-projector, audio levels of the various media and microphones, 
and electronic projector-screen controls. Besides the two built-in computers (Mac and PC), the following 
presentation equipment is included in these rooms and integrated through the Instructor Console with the projection 
system: VCR, DVD player, LaserDisc player, and Document Camera. Each piece of media equipment can operate 
stand-alone (through the touch screen controls), or in conjunction with a selected computer. In addition, an audio - 
cassette machine is available for recording lectures or presentations when necessary. 

The Level II classrooms introduced a major design shift regarding what equipment would be made available 
in these classrooms. These Level II rooms are equipped with "laptop only” presentation systems, meaning that they 
do not include any built-in computers. This was decided for cost and on-going maintenance issues, but also to 
introduce the concept of instructors having all course materials accessible in a portable format and self-contained on 
his/her laptop computer. We wanted to encourage the notion of mobile faculty members who could be flexible about 
the location "where" they taught, because all the required class resources are readily available and always configured 
in the most appropriate manner for each practitioner. These classrooms have a basic Instructor console that contains 
the plug-ins for interfacing with a personal laptop computer, and a smaller B/W "touch screen" for controlling the 
built-in VCR, DVD player and Document Camera. Because these Level II presentation systems are placed in smaller 
rooms, no microphone is provided for voice amplification, but there is an audio system for selected media playback. 

During the summer of 2001, we began the planning phase of a project to create and install a baseline 
presentation system in every General Education classroom on the MSU campus. These Level I rooms are "laptop 
only" presentation systems similar to Level II, but do not include a touch-screen control panel. A simple 
switch/button interface allows the presenter to select between a personal laptop computer and the other media- 
equipment. These Level I rooms are designed to serve as multimedia "docking stations” for each user's laptop 
computer which should include a DVD and CD-ROM player. In August of 2002, we completed the installation of 82 
classrooms in 12 different buildings. This provides multi-media capabilities in every General Education classroom 
(102) throughout the MSU campus. 
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Our intention was to make this equipment as uniform and simplistic as possible, in order to promote ease of 
intuitive operations and facilitate maintenance. Already, we have noticed a favorable impact on faculty instruction 
and student learning on our campus. The equipment is readily available, omnipresent and has a standard interface 
that is common to most rooms. This has greatly improved equipment operations and has encouraged new users to 
integrate multimedia resources into classroom instruction in a seamless and user-friendly fashion. One major 
observation is that faculty are liberated from being classroom or location-dependent to access certain electronic 
materials. Furthermore, instmctors are feeling flexible, mobile, and comfortable knowing that important course content 
is constantly available and tucked under his/her arm in the form of a personal laptop computer. 

Opportunities for Change 

Besides these existing conditions and further developments at MSU, other events were occurring that 
provided additional stimulus to re-examine the technology plans of our institution. The MSU College of Business 
(COB) began to move forward with a plan to require that all students admitted to their program purchase or lease a 
laptop computer. The laptop initiative was based on survey information COB compiled, as well as feedback from 
potential employers and private-sector partners. In conjunction with this initiative, MSU was also working with a 
regional, wireless-phone service provider to create a wireless distribution system on the campus. In partnership with 
Midwest Wireless Communications, this was accomplished through the installation of a digital 800-megahertz cellular 
communications site provided on the MSU campus. This cell site allowed MSU to have robust cell phone 
connectivity, including a portal to SMS (Short Messaging Service) and WAP (Wire Application Protocol) 
capabilities. Because of this existing infrastructure and the creation of the research based Institute for Wireless 
Education, Nokia phones selected MSU to become its model wireless campus in North America. This provided free 
phones and services to MSU students, including a WAP development server to create and test applications. In 
return, MSU serves Nokia as a research and test-bed site for cell phone applications, content, and services. 

These circumstances provided an opportunity for MSU to investigate other approaches for integrating the 
capacity of wireless communications. Because of the interest in the COB laptop program, momentum was generated 
to create a campus-wide wireless network that could provide mobile connectivity for laptop computers equipped with 
wireless Internet cards. In January, 2000 MSU began to install a wireless ethernet infrastructure utilizing products 
based on the IEEE 802. 1 Ib DSSS (direct sequence spread spectrum) standards. In the beginning, MSU experimented 
with Lucent WavePoint II access points (10) and Apple AirPort base stations (60). Initially, the Lucent products 
were chosen because they were the premier solution at that time. However, when the Apple AirPort product became 
available it was discovered that these wireless hubs could be configured to coexist with the Lucent products and at 
one third the price. The Apple AirPorts were extensively tested and these hubs performed well for the MSU 
configuration. MSU installed a variety of indoor and outdoor external antennae to maximize coverage across the 
campus. The technology solutions that were selected for implementation were based on the following needs: 

• Simple connectivity to the Internet for laptop users 

• Access to the internet from virtually anywhere a student would be on campus 

• Cost effective way to connect off-campus sites to the network: All Seasons Ice Arena hockey 
Astronomy Observatory (one mile distance), and Mankato Airport computer lab (8 miles away) 

Wireless Learning Environments 

Our initial focus for wireless connectivity was on common gathering areas that received a high amount of 
student traffic. Next, 80 additional AirPort hubs were purchased to expand the wireless infrastructure to as many 
classrooms as possible. Currently, MSU has achieved coverage for 90% of the campus indoor classrooms and 
outdoor common areas. Eor the off-campus connections (e.g. Mankato Airport)-, wireless point-to -multipoint 
connections were installed to replace over-loaded ISDN connections. Although the current system is widespread, 
additional AirPort hubs are being established to improve connectivity in certain dead zones on the campus. Many of 
these zones exist due to location or construction barriers (basements, upper floors, wall structures, or electronic 
interference). We have discovered that overlapping signals from multiple access points makes it difficult to cover all 
areas without some degradation of service. Additionally, interference from other wireless providers affects our 
outdoor coverage, while 2.4GHz phones and other products can disrupt some of our indoor coverage. Moreover, we 
have experienced user-authentication plus security challenges, and we are discovering that existing methods need 
improvement for our current environment. 
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Nonetheless, the evidence of wireless connectivity is pervasive on the MSU campus. In the classrooms, 
students and faculty are not limited by proximity to hard -wired Ethernet ports and web-based content, discussion 
boards, or e-books can be dynamically accessed from any location in that room. We are employing an "audience 
response" protocol that allows faculty to administer on-line quizzes, or content questions in the classroom and 
provide rapid feedback to students, especially in large lecture classes. This interactive system serves as "check-in 
points” for faculty to quickly receive student responses that help ascertain what content areas are being understood, 
as well as what topics require additional discussion because the learners are finding them difficult to comprehend. 

Over the past year, we are observing more evidence of how the wireless connectivity has expanded the 
locations where learning or instruction may occur. We are witnessing more faculty bringing their classes outside, 
either to supply additional exposure to real world content and context (field sites), or to provide a more casual setting 
than the classroom for exchanging information. However, unlike prior learning activities occurring beyond the 
classroom, both the students and the faculty member can still have connectivity to the network and other electronic 
resources through the wireless infrastmcture. Throughout the campus, there are more indications of informal work 
groups gathering around a wireless computer or a hand-held device to collaborate in a variety of settings. Students 
form groups around a wireless laptop in a variety of outdoor settings on the campus, or cluster together near wireless 
devices at couches in the Student Union building. 

It appears that this wireless capacity has promoted spontaneous and casual computing, which in turn has 
stimulated more occasions for social learning and the sharing of information. Increasingly, students and faculty are 
becoming more aware that they have the potential to conduct ad-hoc meetings and learning opportunities in a truly 
anytime/anywhere mode, while still maintaining full connectivity to Internet-based resources. At MSU, we are 
observing a proliferation of ”un-tethered" learning environments that go beyond the constraints of a 4-walled 
classroom. Spontaneous instructional events frequently take place, because accessing digital information is not 
impeded by location-bound constraints. In fact, providing additional access points for technology interfaces is no 
longer a campus concern, that connectivity is mobile and pervasive throughout the wireless airwaves of MSU. 

Mobile Learning Environments 

With the establishment of our wireless infrastructure, MSU embarked on some trial implementations of 
portable hand-held devices to enable learning activities. MSU received a grant award to explore future capacities that 
might expand wireless transmissions. Through a campus RFP process, MSU faculty projects were selected that 
proposed strategies to integrate the use of hand-held wireless tools into their teaching and assist student learning. 
These projects developed case-based educational research opportunities that explore the ways that wireless 
technologies may effectively influence student processing of information. 

The technology tool provided for the accepted proposals is the Compaq iPAQ 3650, which operates on the 
Microsoft Pocket PC platform. These iPAQs are equipped with a wireless network card, full color display, a 206 MHz 
Intel Strongpoint processor, and 32MB of RAM. Over 200 of these hand-held devices (pre-loaded with Microsoft 
Word and Excel) were distributed to selected faculty and their students. We believe that it is essential to place these 
iPAQs in the hands of educational practitioners and learners to experiment with the ways these tools may promote 
more effective instruction and learning. Some of the areas we asked faculty to consider include: 

• What learning activities will most benefit from access to the dynamic communications afforded by 
wireless hand-held tools? 

• How could students improve their learning process by using this technology in and out of the 
classroom? 

• What portions or units of a course have been particularly problematic or non-motivating for student 
learning? 

• How could wireless tools create more exposure to real-world problem-solving experiences that students 
might face in this field of study? 

Presently, there are 8 faculty projects that were intentionally distributed between different College 
disciplines on our campus. These projects seek to explore at a range of applications with a variety of learner 
audiences. In a course with Dr. Steven Case, graduate Computer Science students are creating software applications 
for the iPAQs and possible ways to integrate them with the Nokia cell phones. Dr. Kent Kalm's students use iPAQs 
in clinical and field situations during Athletic Training courses. The iPAQs afford "on-demand" access to 
information regarding appropriate diagnostic or treatment procedures for specific injuries and patients. At the 
Geography Department, Drs. Jiyeong Lee Cindy Miller and Martin Mitchell are collaborating to integrate the use of 
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four emerging technologies: iPAQs, GPS, Internet GIS and wireless communication with a map server. Students are 
accessing Internet Mobile GIS information to determine exact locations and collect GIS soil pollution data that is 
dynamically stored and transmitted back into a master data system. 

In other projects, Dr. Scott Fee's Construction Management students are experiencing real-time field 
simulations and using iPAQs to push and pull information from a web-based database. These learners experience that 
this type of connectivity allows them to make better work assignments, updated orders for materials, and perform 
estimating or scheduling decisions in real time. Drs. Ann Goebel and Lee Anderson are experimenting with distance 
students integrating these tools with cell phones to establish a LAN-line connection for receiving and sending 
information as part of Industrial Design teams. Dr. Robert Widner of the Psychology Department is examining the 
use of iPAQs as a feedback tool in large classrooms. These wireless devices enable students to respond to 
questions, take on-line quizzes in class, receive immediate assessment scores, or anonymously send inquiries. 

This Audience Response project is developing a toolset that allows a lecturer to perform interactive polling 
of an audience. The intention is to allow an instructor to obtain anonymous, real-time feedback regarding the 
student’s comprehension of the lecture material. For example, lectures at MSU are scheduled in 50-minute intervals. 
The instructor organizes the lecture so that the first 20-30 minutes of his/her presentation provides in-depth 
explanations for a rather complex concept. The instructor prepares several questions to ask the students, which 
indicate how much the learners understand important concepts presented during the class. An instructor posts 
questions on a web-site and the students use wireless-enabled iPAQs to respond to this material. The instructor 
receives a summary of these responses, and uses this information to adjust the remaining class time to clarify any 
misconceptions and review certain topic areas. This anonymous student feedback allows the instructor to respond 
to difficult content areas that have raised the greatest concerns or questions. 

The Digital Image Translation project is developing a mobile system to assist in language translation of 
written information. A typical user scenario would be a business person traveling in a foreign country. Assuming 
the traveler does not read the native language, this person will need assistance translating street signs, billboards, or 
other textual information. Using a digital camera attachment for the iPAQ, the traveler can take a photo of the text, 
identify the specific portion of the image needing translation, and have the image transmitted to a remote server. The 
remote server will then translate the image and send back the correct interpretations back to the end-user’s iPAQ. 

This system has been developed to the point that it is demonstrable using limited textual images. Translations have 
been performed for Arabic and Spanish images, both of which are translated to English. 

The Real Estate Business Systems project is an entrepreneurial endeavor in which students have begun 
developing a prototype system to allow home buyers to browse the multiple listing service (MLS) databases for 
information on homes for sale. The MLS data is presented to the end user in a format suitable to the limited screen 
space available on the iPAQ. This allows users to efficiently navigate the system using the limited data entry 
capabilities of the iPAQ. In addition, the system includes a prototype interface suitable for access via the Nokia 7160 
phones. This expert interface is suitable for Real Estate agents who are familiar with the details of the MLS system, 
but not for typical home buyers. The phone-based interface was developed in a manner that makes it suitable for use 
with iPAQ or other Pocket PC systems 

Evaluation of iPAQ Projects 

The MSU Eaculty investigated a variety of methods for using these iPAQs to enhance the availability of 
instructional materials. Through our Course Management System provider (Ucompass/Wireless Educator), course 
materials were made accessible in a small screen design format. This allowed students to be very mobile and still 
connect with course materials in a truly anytime/anywhere fashion. Another key use of the iPAQs has been as a 
portable communications tool, which allows students and faculty to readily exchange and share information through 
emails, class announcements, chat rooms and discussion boards. The faculty have noticed some achievement gains 
in these courses, primarily due to the increased availability of these modes to interact. We are finding that these 
projects encourage student activity, increase learner involvement, and lead to higher-levels of cognition and 
understanding. The iPAQs provide "just in time learning" opportunities through the rapid transmission of 
information resources or tutorials anytime/anywhere. More evident has been the level of student satisfaction and 
positive attitude towards the courses that have used these portable tools. Students appear to enjoy the increased 
opportunities for feedback with their instructor and dynamic access to test and quiz results. 

As expected, we discovered that the wireles s computing tools enhance certain types of learning activities, 
while other educational applications are marginally affected. Certainly, our experiences have re-enforced the notion 
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that the successful use of any technology must evolve from a sound instructional design process. A thorough 
analysis of critical content and learning objectives is critical for determining the appropriate applications of any 
emerging computing tool. For dynamic access to mission-critical information and point-in-time communications, the 
wireless tools have proven to be a very appropriate technology. For prolonged computing interactions, such as 
retrieving course materials that involve streaming video or high-end graphics - wired ethernet connections work best. 

We found that the hand-held IPAQs performed best as a "viewer” or "window” of information, but it is not 
very appropriate for intensive computer operations. As a mobile tool that provides dynamic access to "bits” of 
information from web-pages or databases for short bursts of time - the iPAQs works very well. Fiowever, these same 
tools are not very appropriate for reading an entire E-book or creating a lengthy PowerPoint presentation. The hand- 
held wireless devices are effective as a technology to gather physical data, create field notes, and push and pull 
information very spontaneously between data resource sites. Students are able to access "just-in-time” information 
for assistance, and send updated materias to web-based databases, which other project participants can access 
immediately. 

Therefore, the most successful applications have been those that used the hand-held iPAQs as a tool to 
gather field data. When observing and recording information at a field-site, the students were greatly enabled by the 
portability of the hand-held wireless tools. Participants could capture written notes and make audio recordings of 
interviews, as well as collect and tabulate physical data about biological or geographic information. The iPAQ 
wireless devices not only provide a user-friendly tool to gather data, but also expedite the sharing and dissemination 
of this information. Learners are able to "push” new findings into web-based data systems, and "pull” information 
from these data inventories for current updates. In this manner, these hand-held devices afford simulated learning 
activities that emulate real world problems involving rapid access and transfer of information. 

Conclusion 

The climate for learning and teaching is changing at MSU due to the impact of a diverse range of 
technology enabling tools. In the wired classrooms, the readily available equipment and connections allows for a 
wide range of media resources to become integrated into lecture presentations. Our campus wireless capacity has 
provided flexible learning opportunities through the omnipresent connectivity to the network. Informal learning 
groups are forming in a variety of settings that are not location-bound to access ports and power supplies. The 
hand-held wireless iPAQs are already demonstrating that highly mobile communications and computing can be very 
productive for data gathering and fieldwork. Faculty projects that are currently using these wireless devices 
continue to explore and discover innovative applications that benefit the delivery of instruction and promote active 
learning experiences. At MSU, having the wireless infrastructure, the portable technology tools, and the capacity to 
provide web-based course content anytime/anywhere has certainly pushed the educational experience beyond the 
confines of the physical classroom. These wired and wireless learning environments certainly appear to facilitate 
collaborative work by student teams, support innovations for classroom delivery, and help enhance the personal 
education and productivity of our participating students. 

User and task analysis are keys to successful implementations of any technology, and this observation was 
borne out time after time with the iPAQ projects. The iPAQs are a part of an emerging toolkit, meaning that they are 
not appropriate for all computing or communication tasks. Yet, at other times, the iPAQs fit the users’ needs, and 
served as effective technology tools for many learning tasks. Our experiences with these hand-held wireless devices 
confirm that this technology promotes productive instructional applications, which enhance and expand student- 
learning opportunities. Our goal is to expand this initiative and obtain additional mobile computing tools. 

It appears that the wireless capacity has promoted spontaneous and casual computing, which in turn has 
stimulated more occasions for social learning and sharing of information. Increasingly, students and faculty are 
becoming more aware that they have an increasing potential to conduct ad-hoc meetings and learning opportunities 
in a truly anytime/anywhere mode and still maintain full connectivity to Internet-based resources. At MSU, we are 
observing a proliferation of "un-tethered” learning environments that go beyond the constraints of a 4-walled 
classroom. Spontaneous instructional events are taking place because the participants are not impeded by location- 
bound constraints to accessing digital information in various formats. In fact, providing additional access points for 
technology interfaces are no longer a real constraint - that connectivity is mobile and omnipresent in the wireless 
airwaves surrounding the MSU campus. 



31 




A Multimedia Database to Improve Teaching with Technology: The "Best 
Practices" Digital Video Project A Brief Report 

Wilheknina Savenye, Arizona State University 
Thomas Bmsh, Indiana University 
Terri Kurz, Arizona State University 
Ann Igoe, Arizona State University 
James Middleton, Arizona State University 

With assistance from: 

Marti Julian, Arizona State University 
Michael Blocher, Northern Arizona University 
Patty J. Horn, Arizona K-12 Center 
John Chadwick, Museum Web Services 
Jean Brush, Arizona State University 
Christine Ziobrowski, University of Redlands 
Ashley Tan, Indiana University 
Barbara Berg, Arizona State University 
Cumali Oksuz, Arizona State University 
Jiahong Li, Arizona State University 
Kathy Perko, Arizona K-12 Center 
Gloria Llama, Arizona State University 
Everett Louis, Arizona State University 
Jean Sutton, Arizona State University 

Introduction 

New teachers during their professional development must gain considerable complex knowledge. Yet as 
preservice teachers they have little experience and so little knowledge about real classroom contexts. At the same 
time our nation is increasingly experiencing an increasing need for well-trained teachers (Bransford, Brown, & 
Cocking, 2000.) 

Studies continue to indicate that teachers feel ill-prepared to use technology in their classrooms (Schrum, 
1999; Strudler & Wetzel, 1999; Topp, Mortensen, & Grandgenett, 1995.) Yet Bare and Meek (1998,) for instance, have 
reported that while 78% of K-12 school teachers and students can access the internet. School administrators, too, 
report that students should have access to and be able to use technology (Brush, 1999; Brush & Bannon, 1998.) 

One way to help educators develop skills in solving complex problems is the use of cases (Stepich, Ertmer 
& Lane, 2001,) or the use of stories (Jonassen and Hernandez-Serrano, 2002.) Although cases traditionally have been 
presented in text form, educators are increasingly turning to the internet to deliver digital resources (Hill and 
Hannafin, 2001,) although such resources should include aids to enable learners not to become disoriented (Hill and 
Hannafm, 1997.) 

Videocases to be delivered over the web have begun to be developed recently; such cases have been used 
to help teachers gain technology skills and knowledge (cf. Hayek & Tanase, 2002;) Talley (2002) notes that cases 
may include teacher interviews. Others have begun to include experts’ reflections as well as lesson plans and 
materials (Savenye, Bmsh, Middleton, Blocher, Horn, Oksuz, Li, Ziobrowski, Tan, Bmsh, Kurz, Chadwick & Perko, 
2002 .) 

Several researchers have suggested that rich, case-based video databases can help teachers (Chaney- 
Cullen & Duffy, 1998; Lambdin, Duffy, & Moore, 1997.) They may also serve as discussion starters (Barab, 
MaKinster, Moore, & Cunningham, 2001.) 
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Several projects have been developed that use videocases to aid teachers. One is the Inquiry Learning 
Forum, designed to help teachers use inquiry practices in science and mathematics (Barab, MaKinster, Moore, & 
Cunningham, 2001.) Another is INTIME (Integrating Technologies into the Methods of Education,) adatabase of 
several hundred short videos specifically designed to help teachers learn to use technology in their instruction 
(Hayek & Tanase, 2002.) INTIME also includes notation tools. 

At Arizona State University a Preparing Tomorrow’s Teachers to use Technology (PT3) grant has enabled 
us to build a database of videocases that focus on technology integration for K-12 teachers and students (Savenye 
et. al.., 2001.) We have partnered with the Arizona AK-12 Center and Northern Arizona University to develop a 
prototype database of “best practices.” 

The Digital Video Database of Rich Cases 

The database provides preservice teachers with sets of instructional materials and methods for utilizing 
these materials. The K-12 Center is hosting the database, so that it is accessible not only to project teachers, but to 
teachers throughout the state. 

Exemplary teachers who integrate technology well are included in the database. They were identified by 
educational technology and methods faculty as well as by mentor teachers. There are currently five prototype cases, 
although ten to fifteen more are currently in development. The teachers worked with our personnel to develop each 
case which includes the videotaped lesson, pre-lesson and post-lesson interviews, commentaries, and instructional 
materials. 

The teachers all use the same, systematically -designed lesson format to ensure that others teachers will 
have enough information to teach the lesson themselves (cf. Dick, Carey & Carey, 2001; Sullivan & Higgins, 1983.) 
Teachers viewing the lessons also learn through the commentaries and interviews more about instructional 
technology as well as instructional design for multimedia, web and distance education (cf. Eastmond, D., & Ziegahn, 

L., 1995; Himmi, A., & Bermudez, A., 1996; McManus, 1998.) 

Format of the Lessons 

Each lesson is built along a “model” or “timeline” of a lesson, which allows for all types of teaching styles 
and philosophies. Preservice teachers access the lessons and lesson materials by clicking on components along a 
timeline that serves as a metaphor for the lessons. Components of the model include: 

— Timeline Components - Eor each lesson the videos include - 
Introduction 

Inquiry and Content Development 
Instruction 

Student Participation and Teacher Eacilitation 
Assessment and Evaluation, and 
Wrap-up. 

Preservice teachers can access any or all components of each lesson and can review any sections they 
wish. 

— Reflective Interviews and Commentaries - 

pre- and post-interviews with each teacher, using a semi -structured interview protocol 
a text -based reflective follow-up by the teachers after they view the final tape of their lessons, 
commentaries by one subject-matter and one technology expert, as well as a peer-expert teacher. 

— Teacher’s Lesson Plan 
— All Lesson Materials 

Example Lessons 

Eifth-sixth grade mathematics. To teach fifth-graders a teacher wrote a lesson on Archimedes Spiral for her 
class of gifted students. In her introduction she helped students recall what they had learned about natural spirals 
earlier. 

She then showed a PowerPoint presentation she developed in which she used illustrations and an animation 
to show how the spiral has been used in history and today. She also developed a Quicktime movie to show students 
how to build their own Arichimedes-type spirals, which formed the basis of the student activity. Students also 
searched the web to answer questions about the spirals and developed their own short presentations. (See Eigure 1.) 
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High-School Language Arts 

A high-school teacher used technology to motivate lower-level students in an English class. She has found 
that building challenging lessons using the web has helped her increase these lower-achieving students’ successful 
completion of this English class. The teacher used a PowerPoint presentation to introduce her students to their four- 
day what she called “focus lesson,” about King Arthur. She suggested web sites students could use in their 
challenge, but students explored beyond these sites as well. Students built cases for who King Arthur might have 
been, and where Camelot might have been. They used “Inspirations” software to plan their case, and then were 
tasked with making a presentation to the class presenting their ideas with supporting rationales. 

Eirst-Second Grade Mathematics 

An elementary-school teacher used a Smartboard to teach her students about finding the area of rectangles 
using coordinate grids and rows and columns. 

Junior-High Technology 

An eighth-grade teacher taught students in her multimedia journalism class how to use digital cameras to 
produce images and then how to edit those images using vector graphics programs, thus working on state 
technology skills standards. 

Implications 

Eormative evaluation is an integral part of the field -based PT3 project. The first three sets of "Best 
Practices" lessons, interviews and critiques are currently being evaluated. Both preservice and inservice teachers are 
reviewing the materials. Surveys and interviews are being used to collect learning and attitude data. 

Our future plans also involve refining the interface and developing web-based delivery systems that will 
allow users to comment on segments of video, by time -code, and share their comments with others. It is hoped that 
these types of discussions will support technology integration instruction in methods courses. 

We also look forward to conducting research on ways to scaffold and support teachers learning about 
effective teaching in math and science and about technology integration. 

We anticipate that access to the "Best Practices" videos, lesson materials, reflective interviews and 
commentaries, both by PT3 teachers and all teachers in the state, will enhance teachers' success in integrating 
technology into their classrooms. Their use of technology will go beyond using computers as tools to allowing their 
students to learn in innovative ways using a full range of technologies. 

(The “best practices” videocase database can be accessed at http://129.219.22.87:89/index2.html using 
username and password “web” and via the Arizona K-12 Center’s web site.) 
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Alternatives for Assessing Learning in Web-Based Distance Education Courses: A 

Brief Report 

Wilhelmina Savenye, Arizona State University 
Introduction 

The World Wide Web is increasingly becoming the technology of choice for delivering courses and 
programs at a distance. The web offers us many means for delivering materials, including those developed using 
multimedia audio, visuals and video, along with providing internet-based telecommunications tools. At the same time 
distance delivery necessitates careful instructional design and use of evaluation to ensure quality of instruction and 
student success in learning (Moore & Kearsley, 1996; Simonson, Smaldino, Albright & Zvacek, 2003.) 

Distance education can be defined in many ways. Rumble (1997,) for instance, stresses the physical 
separation of learner from teacher, and adds that often distance courses use self-study or independent-learning 
approaches. Mclsaac and Gunawardena (1996,) too, emphasize the distance between learners and instructors. 

Moore and Kearsley’ s definition of distance education provides details to engage us in our discussion of systematic 
design and evaluation of web-based learning: 

Distance education is planned learning that normally occurs in a difierent place from teaching and as a 
result requires special techniques of course design, special insfructional techniques, special methods of 
communication by electronic and other technology, as well as special organizational and administrative 
arrangements (1996, page 2.) 

Calvert (1989) broadens the definition of distance education to include learning that occurs not just at a 
distance, but asynchronously. Many students in higher education today, for instance, enroll in distance courses not 
because they live far from campus, but because they prefer the scheduling flexibility such courses allow. 

Web based courses for distance delivery may take many forms. McGreal (2000) describes a thirteen-level 
taxonomy of courses which vary in their degree of distance capability and mediation. Courses may typically, 
however, be fully web-based, or hybrid courses that include some campus meetings, or may include web materials 
only as supplements. 

Whatever form a distance course via the web takes, formative evaluation can improve its instructional 
design, activities and materials. Rumble (1992,) for instance suggests that four main categories of factors to use in 
evaluating the success of distance courses include opportunities for access, completion rates, quality of the output 
and cost-effectiveness. Freeman (1997) adds that baseline data such as data from other similar courses may be 
collected, as can market share information, enrolments, and graduation and placement rates. Thorpe (1988) 
recommends determining the effects of counseling, advising and student guidance. 

Evaluating the success of counseling, advising and guidance provided to students. Kahn and Vega’s 
(1997) suggest additional factors to consider in evaluating web-based courses. 

In the upcoming sections of our paper we will first focus on evaluating web-based materials that form all or 
part of courses or training programs. Then we will discuss methods for evaluating stand-alone web-based learning 
software. 



Evaluating Distance Education Courses 

While the focus of this paper is on assessing student learning as part of evaluating web-based courses, a 
brief review of other data useful in evaluating courses enriches the context of course evaluation. 

Instructional Design Reviews 

The American Council of Education’ s “Distance Learning Evaluation Guide’’ (1996) describes guidelines for 
evaluating distance courses, in an approach that Dick and Carey and Carey (2001) would call “instructional design 
review.’’ In this approach evaluators review factors including the learning design, objectives, materials, technology, 
and the content material. In more data-based approaches to evaluation, data are gathered regarding the effectiveness 
of the course related to various factors, including course implementation, student and instructor attitudes and 
student learning. 
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Course Implementation 

Data may be collected to determine how well the course is implemented. For instance students might be 
asked their perceptions of the technical quality of multimedia materials, and how easy it was for them to access these 
materials, given the various types of internet providers, with related bandwidth, they may use. 

The success of marketing efforts and the students’ success in using, and access to, support services may 
also be measured. In addition to surveys and interviews, archives of use data may be examined. 

Attitudes 

Typically student satisfaction with a web-based course and materials is a concern in evaluating the success 
of the course. Satisfaction of other participants, such as instructors, technical staff, and eventual employers may 
also be determined. Overall satisfaction and perceptions of value are important, however, often just as important are 
student attitudes toward specific aspects of the course. Student attitude data may be used to improve access to the 
course materials, specific materials and activities, etc. 

Attitudes of students, instructors and staff are usually measured using questionnaire and interviews, 
however observations, and transcribed online discussions may also be used. 

Assessing Student Learning in Web-Based Courses 

Student learning is arguably the most important determinant of the success of a web -based course. 
Learning can be measured in many ways, depending upon the course goals, subject matter, institution, students and 
setting. Students in courses such as these, that require considerable student motivation and self-discipline require 
ongoing feedback from assessments. In addition, different measures of learning are valuable to a broad range of 
students (Dabbagh, 2000; Ko & Rossen, 2001.) Multiple measures of performance also enhance security, as 
instructors can compare student achievement across several measures (Ko & Rossen, 2001.) It is often useful to 
collect baseline data to later determine the quality of a course. Pre and posttests can also aid instructors. 

It is also recommended that assessment measures match the varieties of learning outcomes taught in a 
course. For instance. Bloom and his colleagues built a taxonomy of cognitive outcomes (1956.) Gagne (1985) has 
classified varieties of learning as cognitive strategies, attitudes, motor skills, verbal information, and intellectual skills, 
which are further subdivided. 

Gagne (1985,) Dick, Carey and Carey (2001,) Smith and Ragan (2000,) and others have developed research- 
based strategies to help learners master objectives which fall into these categories of learning. For instance, they 
discuss strategies, based upon the social learning theory of Bandura (1969,) for learners to master attitudinal 
objectives. Another view of learning design is that that uses multiple measures and fosters many categories of 
learning is that of DeNigris and Witchel of the University of Phoenix (2000.) They suggest developing lessons that 
include individual study, online lectures, open-ended discussion, group projects and weekly summaries. 

Keeping in mind the many types of learning outcomes that might be measured, in the following discussion 
we will outline various types of assessments that can be employed both in measuring student progress and in 
evaluating the effectiveness of web-based courses. 

Exams 



Many courses, subjects, settings and learners lend themselves to quizzes and exams. These are especially 
useful for assessing background knowledge, verbal information and concepts, but may also be employed to measure 
intellectual skills. Freeman suggests developing assessment systems (1997.) Whether instructor or computer- 
scored, the format of the assessments may include closed-book, or open book exams, portfolio assessments as well 
as learner-negotiated assessments. It should also be determined whether the exam is to be taken at a fixed time or on 
demand. In web-based courses instructors or the development team can determine for how long a test will be 
available, such as 24 or 72 hours only. Also a choice is whether it may be taken once only, a limited number of times, 
or, especially with multiple forms of the exam, until the learner masters the content. It may also be determined that 
students need only retake parts of mastery exams. 

Test security, plagiarism and cheating are issues in most educational settings. Some instructors and 
administrators are concerned that students might share answers to exams or that the person submitting an exam (or 
even an assignment) is not the student enrolled. For these reasons, though exams may be taken online, some 
instructors and organizations, such as in many open universities, have regional centers to which students travel to 
take proctored exams. In the United States many university testing centers are collaborating so students enrolled in 
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distance courses may complete proctored exams at a university center nearby. It could even be arranged that 
someone proctor an exam in a trainee’s work setting. 

Many instructors with large numbers of students and little help choose to rely on computer-scored exams. 
These need not test only verbal knowledge, of course, as sophisticated multiple -choice tests, if well-designed, can 
measure all levels of intellectual skills and deep knowledge. Many web course-development software systems 
include a system for building and scoring exams. Usually items need to be multiple-choice, true/false, matching, or 
short -answer. However, most systems allow for combining these test items with instructor-scored open-ended or 
essay items. 

These tests may take considerable time to develop, however they save the instructor from many hours of 
scoring. Most systems also provide immediate feedback to students, a feature students value. These systems also 
typically provide considerable aggregate data to instructors and designers about student performance and the exam 
itself, which can aid in improving the exam. 

Many good resources exist for test question design. Dick, Carey and Carey (2001,) for instance, provide 
guidelines for types of test items which can be used with various types of behavior stated in objectives, such as 
“identify,” “discuss,” “solve,” “develop, and “generate.” Harrison’s book on designing self-directed learning 
provides suggestions for how to write multiple -choice, true/false, and free-form questions (1999.) 

Self-Evaluation 

Several open learning specialists recommend developing self-assessment questions for students. Race 
(1994,) for instance, describes open learners’ need to learn by doing as a powerful incentive. He then adds that they 
must regularly find out how they are doing. He suggests that self-assessment questions may also be called 
exercises, activities, or self-checks or self-tests. Some designers might consider all of these practice, rather than 
assessment, activities, however they are useful not only to students, but to instructors and designers in evaluating 
student progress. Race suggests helping learners to apply what they are learning by making decisions, extending 
their knowledge, and diagnosing what they still need to learn. He concludes by providing guidelines for writing, 
using and providing feedback on self-assessment questions. 

Thorpe (1988) suggests that learners regularly be asked to monitor their learning, often with the aid of forms. 
This is likely to prove very effective, as learners’ self-study and self-regulation skills are critical to their success in 
web-based courses. Hansen (1998) recommends using self-evaluation with young students, helping them monitor 
their own learning, often by using portfolios. 

Some instructors require students to keep weekly journals or logs, in which they reflect upon what they are 
learning and how they put the course ideas into practice. 

Online Discussions 

Online discussions may be used simply to enhance participation in a course, or for both practice and 
assessment. In our course we use Blackboard, which has an easy-to-use discussion component. We set up 
Discussion Forums for each topic in the course, as well as for socializing (our Online “Cafe,” as suggested by 
Muffaletto, 1997) and in which students “meet” to choose projects and partners. Early in the course the instructor 
moderates the discussion by posting study and provocative questions for each week. Later in the course, students 
do the moderating. We have found in all our courses, that some points must be given to keep the discussions going, 
though many students go beyond the required discussion postings. We require them to post at least once during 
the first half of the week to answer the discussion questions, and then at least once during the second half of the 
week to reply to another student’s posting, (cf. Savenye, 2000.) Conferencing systems can also be used to hold 
discussions. 

Moderating 

By moderating a discussion, students tell us they learn far more than by simply answering the instructor’s 
study questions. Usually they work in pairs, as, again, we have found they feel more a part of the course, if they do 
not always work in isolation. Typically moderators must read the topic readings and resources ahead of time and 
post study and discussion questions by the first night of the week the rest of the class will be discussing the topic. 

A few days after the conclusion of the week’s discussion, moderators post a summary of the key points discussed 
during the week. 
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Individual and Group Projects, Papers, and Portfolios 

In many web courses students write research papers and essays as they do in campus-based courses. They 
typically submit papers either via email or a “digital drop box,’’ in the case of Blackboard 5.0. We have found the 
drop box is very expedient, in that it keeps all submissions from one course together and not mixed in with our 
hundreds of emails, keeps our email account from filling too quickly, and allows us to easily send feedback to 
students. It is important to aid students’ learning by sending them feedback in a timely manner. We have found it 
critical to focus on this, as it is easy to neglect students we don’t see face-to-face, when those on campus are 
clamoring for attention. 

An advantage of a web course is that students’ papers can be posted for other students to enjoy, learn from 
and discuss. Instructors may wish to allow students to revise papers before posting them for others to read. Longer 
papers may be uploaded to course content areas; we have also asked students to post shorter essays in the 
Discussion Forums, to promote deeper discussions on some topics. (For additional views of web-based learning in 
composition courses, see Savenye, Olina and Niemczyk, 2002.) 

In many courses, students are required to develop projects that are not strictly papers. For instance, in one 
of our courses students develop instructional plans. They also may develop instructional software, such as 
multimedia software for learning, using the web, or using development software such as Authorware or Director. 
Other students develop PowerPoint presentations to use in lessons. For any projects, it is a good idea to develop 
very clear directions and assignment guidelines, as well as to make clear rubrics, checklists or rating scales available 
to students at the start of the project, so they know what is expected of them and how they will be evaluated. 

In fully web-based courses projects can be submitted via the web system to the instructor, but it is usually 
beneficial to have students post their projects, if possible, for others to view. Usually these are posted on their 
student web pages, but instructors may choose to post projects to the course web site. 

In all of our web-based courses we support and require that students develop some projects as members of 
groups. Not only does this enhance the social aspects of the course and make it more enjoyable (usually) for 
students, but they may learn from each other as much as from the instructor in this type of setting. 

Many schoolteachers now have extended this idea of group collaboration to having students work with 
others from around the world. For instance, many teachers pair students up with electronic pen pals, not only to 
learn about cultures and language, but about science topics, for instance, in which students collect ecological data in 
two environments and compare them. 

All students in a group may be awarded the same grade, or the instructor may have them submit some 
documentation regarding what each person did. Some instructors also have students rate each other’s participation 
on the team. DeNigris and Witchel provide an evaluation form which students can use to rate the performance of 
their group and of group members (2000.) 

Portfolios are sometimes used to assess student learning in individual courses or as the culmination of their 
learning in an entire degree program. Freeman contends that portfolios are ideal assessments in open learning 
systems, as the student determines to a large degree how to portray his or her achievements (1997.) He suggests that 
managing portfolio assessments can be accomplished provided learners are given clear directions and guidelines and 
that the assessment criteria be made clear to them. . 

Reviews 

In some of our courses, especially those in which students write research papers or essays, students 
indicate they benefit greatly from peer reviews. Usually students send each other these early drafts via email. We 
have found it useful to make this a required component of the course and to provide students with a certain 
percentage of points for participating in these reviews. In some courses such on Composition or Literature, criteria 
may be used to assess the quality of students’ reviews of each other’s work. 

Another type of review is exemplified in courseware reviews. For instance, in our course for instructors who 
wish to learn how to integrate technology into their teaching, students are required to evaluate instructional 
software, videos, web sites and other technology products. We have developed an evaluation form for them to use 
in doing their reviews. This type of project could easily be adapted to other subject areas, such as students in 
advertising courses reviewing ads, art students reviewing art works, or students in design or architecture reviewing 
projects, materials and designs, all of which could be in “real’’ or “virtual” form on the web. 
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Other Assessment Methods 

In web-supplemented or hybrid courses learners are often expected to come to campus regularly to make 
presentations to their colleagues. In fully online courses we have found a version of this can be accomplished by 
having students post their projects for all to view and moderate a discussion about their project. 

“Live” chat in many systems can also be used for students to make presentations. In Blackboard, for 
instance, a student can draw on a white board or go to a web site they developed and/or wish to show for all to view. 
At the same time students can then hold a live discussion. With our system students send the instructor 
PowerPoints to post, but in some systems, students can also make such “live” visual presentations. 

Conferencing systems can also be used to allow students to make presentations. 

In training settings it is becoming increasingly more common for evaluators to be called upon to determine 
the degree to which the web course is successful in helping trainees perform well on the job, not just while they are 
training. With careful planning and diplomacy it is often possible for evaluators to conduct observations of trainees’ 
actual performance on job tasks or sometimes in closely -related simulations. Such observations can be conducted 
using some of the methods of qualitative researchers (cf. Savenye & Robinson, 1996.) 

To our discussion Ko and Rossen add suggestions for using computer and web-based simulations to both 
provide practice and assess learning. They add that role-playing and live or written simulations can be used, as well, 
as online debates (2001.) 

In some open systems learners negotiate their assessments. Freeman recommends management decisions 
are made as to the parameters for these assessments, what can be negotiated, and the recording systems for course 
progress as well as for units of credit (1997.) He adds that instructors and tutors then are responsible for explaining 
the parameters to learners, agreeing together on learner plans, monitoring and recording progress and “signing-off’ 
at completion. 

Conclusion 

It is hoped that this discussion of the wide variety of means to measure student learning in distance courses 
may aid instmctors and designers in developing web-based courses, as well as evaluating their success. The range 
of tools and techniques available for distance learning will continue to expand in the years to come. At the same 
time, the convergence of technologies may enable our courses to become more powerful and more easily delivered. 
We expect our choices for assessing student learning to expand and become more powerful as well. 
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We describe Vizit, a visualization software program we are prototyping to serve two audiences: 1 ) 
learners, and 2) instructional designers. Vizit facilitates learners’ construction of meaning and provides 
instructional designers with tools to create visualizations of concepts. Given Vizit, students and 
instructional designers are able to illustrate causal, sequential, oppositional, hierarchical, 
chronological, mathematical, comparative, and categorical relationships. They can also directly 
represent concepts whose physical attributes have been clearly described in text. 

Theoretical Framework and Background 

Quality instruction provides learners with opportunities for practice with feedback, revision, and reflection 
(Cognition and Technology Group, 1999). Students need to actively engage in learning activities that provide them 
with opportunities to negotiate meaning internally as well as socially, rather than passively receive explanations of 
phenomenon from those in the know. Constructivist learning theory suggests that students need to build 
understanding in the context of mentoring from their teachers who help them find interrelationships among concepts 
(Bransford, 2000; Fosnot, 1996, Reigeluth, 1999). 

Complex concepts presented in texts often contain interrelationships that are difficult for students to 
understand. Braden (1982) reported that inherent structure exists in a verbal text and that rendering the subject as a 
visualization of the text’s structure supports understanding. Text structures include causal, sequential, oppositional, 
hierarchical, chronological, mathematical, comparative, or categorical interrelationships. Learners can use visual 
conventions to organize their thoughts and represent these structures as they construct knowledge of concepts 
(Dodge, 1998). Students who are unable to construct representations of concepts to display their understanding 
probably do not deeply understand that concept. According to Perkins & Unger (1999), in order to constitute a good 
understanding of concepts, students must construct their own mental representations. One strategy to accomplish 
this is student-generated visualization. This strategy has the added benefit that after students visualize concepts on 
paper or on computers, teachers have concrete representations of their students’ understandings that allow them to 
identify misconceptions, assess learning, and provide feedback. 

Visualization is a difficult but powerful study strategy. When students are able to manipulate images during 
knowledge construction, they tend to engage in the meaning-making process and understand and remember 
concepts (Jonassen, 2000). Research provides evidence that students can study efficiently by generating concept 
maps, which demonstrate their understandings of how concepts are related to one another (Anderson-Inman & Zeitz, 
1993). In a meta-analysis exploring the effects and impact of concept-mapping on learning, Horton, McConney, Gallo, 
Woods, Senn, & Hanelin (1993), found that concept-mapping raises achievement and improves attitudes toward 
learning. Also, Cifuentes (1992) and Cifuentes and Hsieh (in press-a, in press- b) found that college students who 
visualized interrelationships among science concepts in their hand written study notes performed significantly higher 
on a test than those students who did not visualize such interrelationships (p= .02, d =. 57). 

In addition to the potential for visualization to facilitate learning, visual conventions need to be part of the 
repertoire of tools that instructional designers use when creating packaged instruction (Wileman, 1993). These 
professionals need to be able to create visuals that clarify concepts and illustrate interrelationships among multiple 
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concepts. Visuals such as diagrams, timelines, tables, charts, and causal chains facilitate understanding for students, 
and therefore, instructional designers need to be able to use them flexibly. 

Computer-Based Visualization 

Computer-based programs such as Inspiration, VisiMap^M^ and AppleWorks Draw and Paint can be used to 
foster visualization. These programs have become pervasive in today’s schools. They can be used to provide access 
to a variety of tools that might help students to visually organize and represent what they know. These tools make it 
easy for students to create, manipulate, and edit shapes and images to reveal interrelationships and information. In 
this capacity graphics programs function as “mindtools” extending and supporting learners in reorganizing their 
cognitive structures during learning (Jonassen, 2000). Computer-generated graphics created by learners offer several 
advantages over pen and paper, such as ease of manipulation of graphic representation, ease of subsequent revision, 
dynamic linkage of concepts, and generation of sophisticated looking graphics by students with undeveloped artistic 
skill (Anderson-Inman, Knox-Quinn, & Homey, 1996). 

Several research studies have provided evidence that computers can be used to support and enhance 
independent study (Anderson-Inman, 1992; Anderson-Inman, Knox-Quinn, & Homey, 1996; Anderson-Inman, 
Redekopp, & Adams, 1992; Tenny, 1992). These studies explored the effects of computer-generated concept-maps 
and outlines on learning. The use of computers in the externalization of students’ knowledge structures enriched 
individuals’ mental models for organizing, retrieving, and using knowledge (Williamson, Jr., 1999). 

Although visualization has been found to facilitate learning, studies indicate that most students are 
unskilled at visualization and require training and experience in visualizing throughout their K-12 experience 
(Cifuentes & Hsieh, in press-b). Application of metacognitive strategies such as visualization requires training, 
discipline, and lifelong practice. Therefore, training in metacognitive strategies in a computer-based learning 
environment can help children to use computers wisely and study more efficiently (Lee, 1997). 

Vizit 

In response to the need for instruction in and experience with visualization, we are developing Vizit, a 
visualization software program that scaffolds students’ construction of meaning while they study. It incorporates 
tools much like those in Inspiration^M and AppleWorks^M. in addition, it offers a palette structure that requires users 
to identify the underlying structure(s) of a text -based concept, provides guidance regarding appropriate tools to 
select for visualizing that concept, and scaffolds decision making while the user visualizes. The program includes a 
database of examples designed to teach users how to distinguish and represent interrelationships within texts. 
Students can use the data-base to develop and practice visualization skills. 

Vizit is designed to serve two audiences: 1) learners, and 2) instructional designers. Learners in grades four 
through college can use Vizit to analyze text and visualize concepts. Instructional designers can use the software to 
illustrate materials for diverse learners in many different types of settings. 

Our Design Process 

For our design of Vizit, we followed a rapid prototyping model (Tripp & Bichelmeyer, 1990). In our planning 
phase we focused on identifying the underlying structures we wanted to support, developing visualizations for each 
of these structures that could be used to represent a variety of relationships within those structures, and determining 
methods for supporting students in identifying underlying structures in text passages and creating accurate 
visualizations. This process led us to a number of design decisions. For example, we decided to include a tabbed 
palette that lists the underlying structures students can visualize so that students are constrained to think in terms of 
these structures. For each underlying structure, we created a set of ''stamps'' that students can use to create 
"instances" of that particular structure. For example. Figure 1 shows the stamps for the Cause/Effect structure on the 
left margin of the screen. These stamps constrain the number of choices students can make in creating their 
visualizations. Finally, we developed a number of scaffolds to guide students in the process of developing 
visualizations. These scaffolds include a "prompts" field, which suggests what students should do next and asks 
questions to promote reflection; a "Show Me How" feature, in which an expert models creating visualizations of 
relationships described in text passages; and an "Examples" feature in which students can look at completed 
visualizations from simple to complex. 
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Figure 1: The Design Document for Vizit, showing the project specifications and use cases. 




The design phase of our process focused extensively on creating an interface for the tool, and on gathering 
and developing text samples for use in the scaffolds. One particularly challenging aspect of this phase was 
determining how to develop design documents that faithfully represented our designs to the project programmer. 
What made this aspect so complicated is that the programmer lives at a distance from us, so impromptu 
conversations were difficult. Our solution was to create an interactive design document in Macromedia Director. In 
addition to presenting a mockup of the interface for the tool, this document includes rollovers of each of the interface 
features that specifies what each does. It also includes a series of use cases. Each use case presents a task that 
users might want to perform, then lists the steps in performing that task. Use cases are effective at communicating 
functionality of the tools as they are used together. In this way, our design document functions as an "interactive 
storyboard," illustrating how various parts of the program appear, specifying how each element functions, and 
offering examples of how these elements are used together to create visualizations. 

Currently, our project is in a prototyping phase. When programming is complete on the initial sections of 
the tool, we will test their effectiveness, make revisions, and then design additional sections. 

Conclusion 
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Vizit software provides students with training and experience in visualization, a metacognitive skill that 
leads to learning. In addition, it provides instructional designers and teachers with tools for creating visuals for 
facilitating student learning. The Vizit interface includes “stamps” and “instances” that can be added to a content 
field to illustrate interrelationships among and within concepts. It scaffolds creation of visualizations thereby 
supporting metacognition and domain expertise (Reigeluth, 1999). Visualizers gain conceptual understanding (what?) 
and theoretical understanding (why?). With Vizit, students and instructional designers illustrate causal, sequential, 
oppositional, hierarchical, chronological, mathematical, comparative, and categorical interrelationships. The tool 
requires students and designers to address the question, “what is the underlying structure of a given text or written 
concept?” It then scaffolds their cognition through the process of clarifying the answer. 
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This paper will provide an overview of strategies to promote collaborative learning in online 
communities from the perspective of the designers or community facilitators, particularly the strategies to 
help establish and strengthen the connections between community members. Related theories and 
research findings will be reviewed briefly and the emphasis will be given to the demonstration of such 
strategies in existing online communities for professional development, especially how they are used by 
designers to achieve the desired goal of connecting members to construct knowledge collectively in the 
online communities for professional development. 

Introduetion 

One major difference between online communities for professional development and other virtual 
communities is that they are mainly concerned about how members will gain some advances related to their 
professional field by constructing knowledge through online interaction. The created knowledge is meaningful not 
only to the members in the virtual community where it was created, and but also to all the professional practices and 
knowledge base of the field. This is achieved with the joint effort of community designers and members. Specifically, 
online knowledge construction is one of desirable results of the process where designers provide various support to 
members to ensure the success of human-computer interaction and human-human interactions (Barab, in press). The 
latter has been given more attention (Barab, 2001) in that it has been realized that it is in such interactions that 
members conduct task-driven or social-emotional communications (Hare & Davies, 1994), create their identities and 
build their online social networks. These interactions significantly impact the growth of the group and the 
construction of knowledge. 

The paper will focus on design strategies to support collaborative learning in online communities for 
professional development. Special attention is given to strategies to support newcomers, help establish connections 
between old-timers and newcomers to create “apprenticeship” learning opportunities, and to facilitate members in 
locating information for their practices or personal knowledge construction. Examples from several online 
communities for professional development, such as TAPPED IN, Math Eomm, and MediaMOO as well as examples 
from other virtual learning communities are used to illustrate how the strategies are realized in those communities. 

The strategies include: (1) the use of metaphorical design to create a shared context; (2) the use of virtual objects to 
establish connections; (3) designers’ support for visitors and novices; (4) designers’ support for communication; (5) 
designers’ support for knowledge construction. 

Metaphorical Design: Creating a Shared Context 

In physical communities of practice, the socialization structures such as buildings, plans, and schedules 
have been so integrated into the traditional learning process that they become tacit and almost “invisible”. A 
challenge for the designer is to create a shared context for members with diverse background. One approach to 
address this issue is to use metaphorical design and making the socialization structures transparent (Dirckinck- 
Holmfeld et al, 1999). Eor example, designers of TAPPED IN ( www.tappedin.org) use a building metaphor to create a 
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sense of time and space. The floor plan (see Figure 1) provides a layout with objects that members are familiar with, 
such as whiteboard, elevator, cafe, and library. A corollary of such design is that it is intuitive for members to 
navigate and natural to define their roles since the virtual world bear great resemblance with the physical world that 
members encounter in their daily practices. 




Figure 1. Floor Plan of TAPPED IN (from www.tappedin.org) 



Similarly, like any other organizations in physical settings, online communities can have their own event 
calendars and newsletter to inform members of coming events of the community to provide a shared context. 

Another strategy that is common in many communities is that members are allowed to inherit or even create 
objects. These objects serve important purposes that will be described in detail in the following section. 

The Use of Virtual Objects to Establish Connections 

Designers can use virtual objects in various ways to establish connections in the community. In some 
communities, artifacts such as virtual objects are provided by the designers and members can make a copy and edit 
them while in other communities members who get permissions from community administrators can program their own 
objects (Bmckman, 1995, 1997, 1998). The use of virtual objects serves several purposes. First, virtual objects 
especially pets, personal items give members companionship and alleviate their loneness since they can interact with 
objects before they establish connections with other members. For instance, in Tapped In, there are commands 
available for members to look at an object, give an object to somebody, take an object from another person or simply 
drop the object. More importantly, they are important in building connections. For instance, old-timers may have their 
own virtual offices that they can introduce newcomer to. It also can be a place where they give virtual classes to their 
students or have workshops for their colleagues. Sometimes such offices can gradually become public places where 
many members/friends have gatherings or where community events are conducted. These offices, not only help 
building the relationships between community members, but as the offices become part of the community 
possession, the bond between the member(s) who created them and the community has been strengthened. The 
process can be more rewarding if some members join the designers in creating objects for the community such as 
member directory, as documented in MediaMOO project (Bmckman, 1995). 

Thus, virtual objects can be a powerful tool for designers to build relationships between members and 
strengthen the connections between members and their community. 

Support for Visitors and Novices 

Every member in the community starts as a visitor and then a novice before they become “formal” members 
of the community. This means that the assistance that designers provide to support “peripheral participation” (Lave 
and Wenger, 1991) greatly affect the growth of the community. The following are various supports available for 
visitors and novices. 

Eor visitors and newcomers of a community, it is sometimes overwhelming to find some many newsgroups, 
mailing lists, and Web-based discussions that are conducted in them. Devices for them to find their way to their 
subgroups should be available. For instance designers of Mathforum ( www.mathfomm.corri) provide “i “ at the end 
of each discussion group for newcomers’ reference. 
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Discussion Groups 






t. math, undergrad _i_ 
amte _i_ 

ap-calc _i_ 
ap-stat _i_ 
calcTreform i 



kl2. ed. math 
mat he dec _i_ 
mathed-news _i_ 
mathedu _i_ 
math-1 earn _i_ 
mathqa _i_ 

math-history- list _i_ 



Figure 2. Screen shot for discussion groups ( http://www.mathforum.com/discussions/ ) 



If the user does not know what the discussion group is about, they can click on the “i “ button for 
information on the purpose of the discussion group, how it can be accessed, whether it is moderated, as well as 
where to obtain detailed information on the discussion group, as indicated in Figure 3. 

About kl2. ed. math 



Public Discussions at the Hath Forum 



The focus of kl2. ed. math is on general math teaching questions. This 
group can be accessed in several different ways: 

1. As a Web-based discussion 

Read, search, and post messages within the Forum’ s 
kl2. ed. math Web discussion . 

2. As a Usenet newsgroup 

View kl2. ed. math within the newsreader of your choice. 

As of 1 August, 1997, this newsgroup is moderated. Please read the 
kl2. ed. math charter for the rationale for converting to moderated 
status, purpose of the newsgroup, policy of how a message gets posted 
to kl2. ed. math, and additional resources and information. 

Figure 3. Information on the discussion group of kl2.ed.math from 
( http://www.mathfomm.com/discussions/about/kl2.ed.math.html ) 

Such tools can be built into the system, which makes it convenient especially for newcomers or visitor to 
check out the community. 

There are other tools that can be used to help newcomers or visitors to get connected with more established 
members of the community. For instance, in TAPPEDIN, once you log in as a member or a guest, you can find out 
who is in the same place (cafe, library, your virtual office) and brief information on them, if he/she is a member of the 
community. The following is a screenshot of information I found out about BIB, a volunteer at help desk, when I 




registered as a guest and clicked on the “ ” icon from the right (Figure 4. Note: permission was obtained from 

BIB). Novices who would like to know more about the old-timers they come across can click on the links to their 
homepage to find more about their work and life and start to build personal relationships offline. In some cases, 
members’ picture is available and communication between novices and old-timers can become personal and 
interesting. Once apprenticeship (Collins, et al., 1989) has been formed between new members and old-timers, it is 
more likely that novices will come back for more information or to connect to other members of the community. 
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Connected for 2 hours, 7 minutes, and 28 second; 
Idle for 29 seconds 



Link to: httpc//www. backflii^com/menibers/b.iberquist 
BJB is an art teacher at the Loysville Youth 
Development Center, a residential male juvenile 
correctional facility in central Pennsylvania. She was 
involved in the planning of the Civil Rights Forum held 
in Hay, 1998, has led several discussions at TAPPED IN, 



Figure 4. Information on an old timer of the community (www.tappedin.org) 




Making communication commands from day 1 available for visitors and novices is always a strategy to help 
them to find assistance from other members and thus facilitate relationship building in a community. This will be 
elaborated in more detail in the following section — support for communication. 

Support for Communication 

In virtual communities, most of the communications are text -based and it is difficult to get “personal” 
without sounds or pictures. To facilitate collaboration between members, it is necessary for designers to have a 
variety of communication channels as well as other devices that help members express their emotions and feelings in 
their online interaction. 

Various channels of communication should be provided by the community, including asynchronous 
communication such as threaded bulletin boards, synchronous communication such as virtual chat, as well as 
traditional communication modes such as organizational conferences. Also commands for private and public 
communication should be made available. The commands in Table 1 are used in Tapped In and the mode of the 
communication (private or public) and how they are used for communication based on my personal experience as well 
as what other researchers have found about communication in MOO environment (Cherny, 1999). 



command 


Private or Public 


Used for Talk... 


/Say <text> 


public 


Within a room 


/emote <action> 


public 


Within a room 


/To <person> <text> 


Public 


Within a room 


/whisper <personxtext> 


private 


Within a room 


/page <personxtext> 


private 


Across room boundaries 


/think <text> 


public 


Within a room 


/sign <text> 


public 


Within a room 



Table 1. Basic MOO communication commands 

Although most of the available communication commands are for communication within the same room, 
/page <personxtext> command can be used to talk to members at other locations. Also the command “/who” will 
provide a list of members who are online, for how long they have been connected, whether they are currently active 
online, as well as where to locate them. It should be noted that in Tapped In, the communication tools are available 
from day one and they are conveniently placed where users especially novices need to refer to from time to time for 
their online interaction. 

Although commands such as /emote <action> enables members to show their feelings with behavior, 
graphic representations of facial expressions called “ emoticons” have been widely used to express emotion and 
feelings in computer-mediated communication. Research indicated that though emoticons were not weighted as 
much as verbal content by the users, they had the same impact as verbal content of shifting interpretation in the 
direction of the negative element when there was a negative message aspect to the emoticons (Walther, 2001). Thus 
such devices to some extent add more emotional cues and “social presence” (Gunawardena, 1995; Gunawardena et 
al., 1997) and reduce potential misunderstanding in communication. 
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Some researchers (McLean, 1999) suggested that support be for “meta-communication”, the communication 
about the process of knowledge building, in addition to the support for articulation of the “substantive” learning 
itself. This is more directly related to devices for reflection and support for knowledge construction, which is the 
focus of the following section. 

Support for Knowledge Construction 

Instances of knowledge construction occur in the most common discussions and chats. In online 
communities, most of communication is asynchronous and thus provides members with sufficient time to 
contemplate, speculate and reflect, which makes the quality of interaction superior to that of face-to-face interaction 
(Jonassen et al, 2001). In addition, various other devices should be employed by designers to foster knowledge 
construction that originates from valid argumentation which derives from logical processes and is based on 
established standards for argument form and content (Derry et al., 2000). The following is a review of such strategies 
that have been practiced in some communities. 

One effective strategy in helping learners produce quality interaction is to provide scaffolding to guide 
online discourse. This strategy has been successfully implemented with young learners at LearningForumTM 
(Scardamalia & Bereit, 1996 from Jonassen, 1999) by having students select various names (my theory, I need to 
understand, new information) for preparation of their messages, as illustrated in Table 2. These names 'serve as meta- 
cognition tools for learners to reflect their thinking process and meta-communication (McLean, 1999) for students to 
communicate with each other their thinking. 



Problem (Andrea): 

My Theory (Andrea): 

My Theory (Jane): 

My Theory (Sue): 

I Need to Understand 
(Andrea): 

New Information (Andrea): 



I Need to Understand 
(Andrea): 

My Theory (Andrea): 

My Theory (Sue): 

I Need to Understand (Sue): 
My Theory (Andrea): 



How does a cell function? 

I think a cell functions by oxygen coming into the cell and the cell then can do its work 
by breathing. 

I agree with your theory, but when the cell functions I don’t think it is breathing. I 
think that the oxygen you’re breathing in is doing it. 

I think a cell functions by the “things” inside itself, (organelles) 

How does the oxygen get into the cell, if the cell really does breath oxygen? 

I found out that the cell takes food and oxygen in through the membrane. This 
happens regularly. The cell then changes the food and oxygen into energy. It uses the 
energy to do its work. 

How do the food and oxygen get to the cell’s membrane? 

I think there are very small tubes that lead to each other and the food and oxygen go 
down those tubes and into the cell through the cell’s membrane. 

I disagree with your theory, Andrea. I think that the oxygen and food go into the cell 
automatically as a daily process. 

What does the oxygen do when it gets to the cell? 

This is what I think the oxygen does when it gets to the cell. I think that the oxygen 
goes into the cell through the membrane and then it goes to the nucleus where it is 
turned into energy. 



Table 2. CSILE discussion note, from Jonassen (1999: 129) 



Along the same line, scaffolds specific to teachers’ professional development can be developed based on 
research results. It was found that stories and narratives are good sources for teachers to exchange ideas and learn 
from each other. Thus a case-based database can be created to collect stories and help community members to share 
their experience. Figure 5 provides an illustration (Jonassen, 1999) of how this may look like. 
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Layout ttl 







Records: 

1 

Unsorted 



Type of Confeicftce 
Cbssioofn Racemert 
Reason for Conference 
Goal of Conference 
Rai 



iReguiariy Scheduled j 

iLearning Support j 

iBehavioral I 

ifeacherwanted control over parents during conference i 

jfeacher insists on parents not speaking until end oif conference ] 



Res<* of Conference iDisasterous, parents walked out. 
Reflection by Tescher [oe more ilexible and interact with parents 



il should have “included" the parents in the conversation from the beginning. 

/itefnainre Teacher Action : 



Story II was a newteacherand was on guard for my first set of conferences. I was 
idetermined to show who was "boss", i did all the talking and asked the 
iparents not to say anything until end. Each time they interupted me, I asked 
ithem not to. By the end of the conference, they were clearly put off. They didn't 
iask questions, and got up and walked out. 



Figure 5. Database of teacher stories, from Jonassen (1999:161) 

Once the database is in existence, knowledge maps or concept maps (Hewitt & Scardamalia, 1998) should be 
created by subject experts or assigned group members to assist members in locating information. Such maps give 
members a visual display of both the knowledge created by the whole community and the knowledge built by 
different groups in the community and members are more likely to recall more main ideas in maps (Hall et ak, 1992, 
1999; Rewey et ah, 1991) since such maps “make the macrostructure of a body of information more salient.” 
(O’Donnell, Dansereau, & Hall, 2002). 

Finally, it is beneficial to build the online community in partnership with other existing communities, being it 
virtual or real (Barab et ah, 2001). This way the new community can use the established networks in the subject field 
and it can be easier for the community to be an integral part of the existing systems of the professional field. 

Conclusion 

There are various devices that are in practice for designers to support online collaborative learning in online 
communities for professional development. Table 3 is a summary of strategies discussed in the above sections. 



Aspects 


Example for specific strategies 


the use of metaphorical design to create a 
shared context 


Floor plan (Figure 1) 


Community calendar, newsletter 


the use of virtual objects to establish 
connections 


Pets, virtual offices, member directory created by volunteer members 


designers’ support for visitors and 
novices 


Devices for novices to check out the community (Figure 2 & 3), make 
the communication commands handy, make information on old-timers 
easy to access (Figure 4) 


designers’ support for communication 


Multiple channels for communication, allow both public and private 
communication (Table 1), use of emoticons, meta-communication 


designers’ support for knowledge 
construction 


Scaffolds (Table 2), case-based database (Figure 5), knowledge 
map/concept map, partnership with other existing communities 



52 



Table 3. Summary the strategies discussed 

The above strategies are only some examples of different support that can be provided by community 
designers to help connect community members and facilitate the process of collaborative learning and knowledge 
construction. With the emergence of more successful online communities for professional development, it is expected 
that researchers and designers, with the input from community members, will find more strategies that can be used by 
designers to support collaborative learning in such communities. 
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GIS in Education: An Examination of Pedagogy 

Aaron H. Doering 

University of Minnesota 
Abstract 

Geographic information systems (GIS) have been identified as one of the most critical and important 
software programs for implementing computer-based technology in social studies and science education. 
Much of the research about effective GIS integration is “ garnered from intuition. ” This study will present 
the results of three different teaching methods based on the Jasper Woodbury Problem Solving Series and 
provide a model for successful GIS implementation into any preservice teacher education program. 

GIS in Education: An Examination of Pedagogy 
“No exemplary models for integrating GIS into preservice 
teacher preparation programs exist.” 

Bednarz & Audet, The Status of GIS Technology in Teacher Preparation Programs, (1999) 

The effective teaching of the National Geography Standards has been the focus of K-12 social studies 
curricula throughout the nation. It has been identified that in order to effectively teach the standards, teachers 
require a clear understanding of a geographic information system (Bednarz, 1995; Sui, 1995). A geographic 
information system (GIS) is software that allows a user to store, retrieve, manipulate and display geographic data 
about any place in the world (Environmental Systems Research Institute, 1998). Even though such an understanding 
of GIS is necessary, it has not been adopted in the K-12 American classrooms at a rate that the National Science 
Eoundation, the Environmental Systems Research Institute (ESRI), and geography educators had once hoped 
(Environmental Systems Research Institute, 1998; Eitzpatrick, 2002). Eitzpatrick (2002) noted that ESRI’s goal in 1992 
was for K-12 educators to be the largest single group of users by 1996, a goal still not achieved. The key reason for 
this slow pace of GIS integration, according to Bednarz and Audet (1999), is that no “exemplary models for 
integrating GIS into preservice teacher preparation programs exist’’ (p. 65). 

In response to this lack of exemplary models for teaching GIS, the purpose of this study is to develop and 
research the effectiveness of three GIS instructional models for university-level instructors to use within preservice 
teacher education courses. 



State of Current Teaching of GIS 

Bednarz and Audet (1999) have identified three main reasons that current approaches to teaching GIS in K- 
12 classrooms have not been effective: teachers inadequately trained to use GIS, a lack of teaching models to guide 
their pedagogy, and preservice teachers education programs that do not teach GIS in “a meaningful way” (p. 64). 
Teachers who are adequately trained to use GIS in their classroom endure major frustrations and obstacles in their 
efforts to successfully integrate technology into their instmction (O’Neill, 1995). Eitzpatrick in communication with 
Audet and Paris (1997) has estimated that fewer than 20% of GIS software packages are being used in effective GIS 
teaching. Many teachers are excited about using GIS, but their motivation dissipates when they encounter technical 
and pedagogical problems. Thus, argues Brownwell (1997), the effectiveness of current approaches recommended 
for GIS should be questioned, and preservice teachers with time and access to university computer labs should 
become proficient in both the technical and pedagogical approaches to teaching GIS in the classroom. Bednarz 
(1999) not only calls upon teacher education programs to more effectively prepare preservice teachers to implement 
GIS-based curricula, but upon the research community to provide more than anecdotal evidence to support the claims 
that GIS can enhance students’ learning. 

This study, in response to BrownwelTs (1997) and Bednarz’s (1999) appeal for the development of an 
effective GIS teaching model for K-12 classrooms and for university preservice teachers, develops and researches 
three GIS teaching models for preservice teacher education programs. The GIS pedagogies are based on three 
instructional models employed by teachers of the Jasper Woodbury Problem Solving Series published by the 
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Cognition and Technology Group of Vanderbilt (CTGV). To help students develop effective problem solving skills, 
the CTGV encourages teachers to use anchored instruction which “provides a way to recreate some of the 
advantages of apprenticeship training in formal education settings involving groups of students’’ (CTGV, 1990, p. 6) 
by creating learning environments that emphasize scaffolding and generative thinking (CTGV, 1992). These 
instructional approaches are the same ones that many educators have identified as necessary for successfully 
teaching GIS to learners at all age levels (Bednarz, 1995; Keiper, 1999). Therefore, the goals of the Jasper Woodbury 
Problem Solving Series and GIS are for students to “learn to become independent thinkers and learners rather than 
simply become able to perform basic computations and retrieve simple knowledge’’ and to develop their ability to 
“identify and define issues and problems on their own rather than simply respond to problems that others have 
posed’’ (CTGV, 1990). 

The three instructional models CTGV (1990) identified for teaching the Jasper series are: “Basics First, 
Immediate Feedback Direct Instmction,” “Structured Problem Solving,” and “The Guided Generation Model.” The 
first model, “Basics First, Immediate Feedback Direct Instruction,” focuses on the order in which content and tasks in 
a curriculum are presented. This model is grounded in a “extreme reductionist view that all components of a skills 
must be mastered before the components can be assembled into the skill they comprise” (CTGV, 1990). The second 
model, “Structured Problem Solving,” focuses on the importance of learners making errors and struggling with a task. 
This model is based on the premise “that errorless learning is ideal; at the other is the assumption that important 
lessons of learning occur only when students make errors or reach impasses and are then helped to correct their 
initial misconceptions” (CTGV, 1990). The third model, “The Guided Generation Model” is grounded in the role of the 
teacher in the learning process. This can span the range of “authoritative provider of knowledge to a resource who 
at times is consulted by the students and at other times can become the student whom others teach” (CTGV, 1990). 

In addition to developing and researching GIS instructional models, because GIS use is grounded in a 
learner’s ability to successfully navigate and learn with a spatial perspective (Audet & Paris, 1997), this study will 
analyze the effect of learners being field dependent or field independent on GIS learning. Field dependence and field 
independence is an established spatial ability cognitive style that correlates with the ability to successfully perform 
in a computer-based instructional environment (Riding & Cheema, 1991). 

Statement of the Research Questions 

This study is designed to answer the following questions based on the development and research of GIS 
teaching models for university instructors to implement within their preservice teacher education programs: 

To what extent will preservice teachers, who are taught with a basics first (BF) model of instruction, be able 
to perform on a GIS basic skills exam (i.e., a test of procedural knowledge) and problem solve (i.e., a test of GIS 
cognitive engagement) in a GIS environment immediately after instruction and two weeks later? 

To what extent will preservice teachers, who are taught with a structured problem solving (SPS) mo del of 
instruction, be able to perform on a GIS basic skills exam (i.e., a test of procedural knowledge) and problem solve (i.e., 
a test of GIS cognitive engagement) in a GIS environment immediately after instmction and two weeks later? 

To what extent will preservice teachers, who are taught with the guided generation (GG) model of 
instruction, be able to perform on a GIS basic skills exam (i.e., a test of procedural knowledge) and problem solve (i.e., 
a test of GIS cognitive engagement) in a GIS environment immediately after instruction and two weeks later? 

Does field articulation impact learning within the three pedagogical approaches - basics first (BF), 
structured problem solving (SPS), and guided generation (GG)? 

Methods 

In order to provide a context for the present study, descriptions of the research methods are organized into 
several different sections. They include descriptions of the subjects, materials, design, research setting, procedure, 
data scoring, and statistical analyses. 

Subjects 

The pool of available subjects in this investigation consisted of 160 volunteer university preservice teachers 
(hereafter participants) who were enrolled in the post-baccalaureate teaching program at the University of Minnesota. 
The participants were recruited from seven sections of the EdHD 5007: Technology for Teaching and Learning 
course. 
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Materials 



The materials for this study consisted of the geographic information system software entitled Arc Voyager, 
three web-based modules with strict pedagogical guidelines - “Basics First, Immediate Feedback Direct Instmction,” 
“Structured Problem Solving,” and “Guided Generation,” the Group Embedded Figures Test (GEFT) to measure the 
participants’ field articulation, the project exams - “The Case of the Missing Ship” (ESRI, 1997) and “Magic Dan’s 
Extreme Sea & Ski Resort” (ESRI 1998), and a response confidence survey/ demographic questionnaire. 

Instructional Approaches Materials 

Three instructional methods were used in this study to identify the best approach for GIS learning and to 
develop a model to guide future teachers who use GIS in their instruction. Each instructional method was based on 
suggested instructional approaches to teach the Jasper Woodbury Problem Solving Series from the Cognition and 
Technology Group at Vanderbilt (CTGV, 1992). The CTGV named their instmctional methods “Basics Eirst, 

Immediate Eeedback Direct Instruction,” “Stmctured Problem Solving,” and “Guided Generation.” The “Basics Eirst” 
method focused on participants acquiring the basic subskills and subconcepts prior to being presented a complex 
problem. The “Structured Problem Solving” method focused on participants receiving instruction that minimized 
errors and reduced confusion and the “Guided Generation” method introduced complex tasks at the same time as the 
basic subskills and subconcepts. The materials developed for each method of teaching GIS were as follows. 

Basics Eirst, Immediate Eeedback Direct Instmction Materials 

The materials developed for the “Basics Eirst” model were developed for a directed method of instruction 
where the instructor is the purveyor of knowledge while making sure students acquire the necessary GIS subskills 
and subconcepts. The “Basics Eirst” GIS model was a web-based module developed with Macromedia Dreamweaver 
that included text and graphics to illustrate the main GIS subskills and subconcepts. This module was designed to 
help learners build their basic cognitive foundation in the use of a geographic information system prior to 
participating in the project exams. All “Basics Eirst” sections were decontextualized to emphasize the basic skills. 
Therefore, basic GIS skills needed to complete the project exams were covered in this module before they practiced 
the task themselves and before they independently took the project exam. 

Stmctured Problem Solving 

The teaching materials developed for the “Structured Problem Solving” model were developed so 
participants could minimize their errors without confusion while working only on activities that would hopefully 
generate the correct result. A worksheet with ten complex practice questions 

( http:///www. geo graphveducation.com/gisweb/structuredindex.html ) was used that required participants to use basic 
subskills and subconcepts in parallel with learning the ten sections in the “Basics Eirst” web-based module. In order 
that participants could minimize their confusion within this module, Quicktime video tutorials 
( http://www.geographveducation.com/gisweb/structuredindex.html ) were developed. The QuickTime videos 
included graphics, animation, and narration that provided a web-based tutorial for students to successfully complete 
the ten complex practice questions. By providing a QuickTime video, students could play, stop, and replay the 
video, reducing confusion and minimizing errors while completing the practice questions. 

The Guided Generation Model 

The “Guided Generation” materials were developed to emphasize generative learning, anchored instruction, 
and scaffolding (CTGV, 1992) . The materials in this module consisted of a video news clip 

( http://www.geographveducation.com/gisweb/guidedindex.html) that imitated a news report announcing the “Case of 
The Missing Ship” (ESRI 1998). Within this video, the news anchor provided nine clues that could have been used 
with a geographic information system (GIS) to find the ship’s location. The video provided “anchored” instruction 
for the GIS learners. In addition to the video news clip, in order to scaffold the participants’ learning, a web-based 
module featuring QuickTime videos was developed to instruct participants on four domains needed to solve the 
problem of finding the ship. These domains consisted of the biosphere (people, plants, animals), atmosphere (air), 
lithosphere (rocks), and hydrosphere (water). 

Project Exams 

The participants’ abilities to learn and use GIS were measured by how well they performed on two project 
exams and two GIS basic skills exams. The project exams measured how well the participants used GIS to identify a 



57 



given location on a map when given nine clues. The exam questions were set in an authentic situation that the 
students could relate to in their everyday life. Both project exams have been used extensively in GIS training 
throughout the nation and have been found to be valid and reliable (Fitzpatrick 2002). Fitzpatrick (2002) noted that 
the project exams clearly measure how well students learn and use GIS at all age levels while providing a consistent 
measurement over the last four years. 

A GIS basic skills exam was also used to assess the participants’ ability to understand basic GIS procedural 
tasks. The basic skills exam consisted of ten multiple -choice questions (see 
http://www.geograt)hveducation.com/gisweb for the exam). 

Response Confidence 

Kulhavy, Yekovich, and Dyer (1976) found that during an exam, seldom does a test taker know their answer 
is completely correct or incorrect. In fact, they found that the students create a “hierarchy of confidence” regarding 
how correct their answers are (p. 522). Therefore, because the participants could possibly guess and receive a high 
score when taking the project exams, how confident the participants were in their answer was measured by 
participants ranking themselves on a five point equidistant Likert scale ranging from 1 (low confidence, random 
guess) to 5 (highest confidence, sure of answer). 

Design 

The design of this study was a quaskexperimental 4X2X2 factorial design. The dependent variables 
included cognitive engagement and procedural knowledge with two levels in each representing the exams 
immediately after instruction and two weeks later. The four independent variables were four different teaching 
methods - Basics First, Structured Problem Solving, Guided Generation, and the Control Group. 

Setting 

The instruction for all GIS methods took place in a Macintosh computer lab. All participants had their own 
computers. All three web-based modules were burnt on a CD-ROM so access time to text, graphics and videos was 
not a problem. The instructor had a Macintosh computer connected to a projector that was viewed on an 8 by 8 foot 
theatre screen for instruction. All modules and videos were available to the participants and were also used for 
teaching by the instructor on the theatre screen. 

Procedures 

Participants who volunteered to participate in the study were divided according to the days they were 
available to have a maximum of 24 participants in the computer lab at one time. The instmction began with each 
student completing the Witkin’s Group Embedded Figures Test (GEFT) to determine his or her field articulation. The 
GEFT procedure strictly followed the GEFT manual (Witkin et ah, 197 1) which required a maximum of 20 minutes for 
the instructor to give the entire exam. Upon completing the GEFT, the students launched Microsoft Internet Exp lorer 
and went to a predetermined bookmark where they answered the demographic questions online. These questions 
were recorded automatically in a database. After the demographic questionnaire, the participants received one of the 
four instructional methods. Of the 160 participants that were invited to participate in the study, 40 were instructed in 
the “Basics Eirst” method, 35 were instructed in the “Structured Problem Solving” method, 35 were instructed in the 
“Guided Generation” method, and 32 received the control group instruction. 

After the instruction, the participants completed the first cognitive engagement exam titled “The Case of the 
Missing Ship” (ESRI 1997). Using the clues from the exam, the teachers wrote the latitude and longitude of their 
answer and recorded their response confidence. Upon completion of the first project exam, teachers completed the 
GIS basic skills exam. The teachers’ delayed-retention was measured when they reconvened in the same computer lab 
to take the second project exam titled “Magic Dan’s Extreme Ski Shop” (ESRI 1998) two weeks later. The teachers 
once again identified the location they believed was the correct answer by writing the latitude and longitude and 
recorded their response confidence. The teachers also completed the second GIS basic skills exam. They were given 
90 minutes to complete both exams. 

Personal Interviews 

Personal interviews were conducted to better understand the participants’ experiences using a geographic 
information system. Eive participants were interviewed for each pedagogical level for a total of twenty interviews. 
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They were asked questions to help the author better understand the benefits and drawbacks of each pedagogical 
model. The interview questions can be seen at http://www.geographveducation.com/gisweb') . 

Data Scoring 

A geographic information system is a spatial environment. Therefore, the two project exams, “The Case of 
the Missing Ship” (ESRI 1997) and “Magic Dan’s Extreme Sea & Ski Resort” (ESRI 1998), were scored based on the 
distance of the participants’ answers from the actual answer location. The distance was measured in total degrees 
latitude and longitude from the correct location. The basic skills tests were graded on the scale of the number of 
correct out of ten. 



Results 

This study was designed to determine the effects of three pedagogical approaches for teaching preservice 
participants GIS and to identify whether or not field articulation has any impact on GIS learning. 

Analyses of Participants’ GIS Cognitive Engagement 

Cognitive engagement (CE) refers to how well the participants solved authentic GIS problems over time. In 
determining the most effective teaching model, a repeated measures analysis of variance (ANO'VA) was computed 
followed by a Tukey One-’Way ANO'VA. All analyses were computed at the .05 significance level. 

Participants’ performances on the CE tests over time were dependent variables with field articulation (EA) 
and group membership (GM) used as between group factors. This analysis revealed significant main effects for 
cognitive engagement (E = 36.075, p < .001) and group membership (E = 6.048, p < .001). Two-way interactions 
between cognitive engagement and group membership (E = 4.476, p < .05) were also found to be significant. No other 
tests were found to be significant. 

The main effect of cognitive engagement over time and the interaction effect of cognitive engagement over 
time and group membership were significant. Therefore, a Tukey One-Way ANOVA was performed to further 
analyze mean differences. Cognitive Engagement exam one analyses found no treatment groups significant. 

However, when measuring delayed-retention two weeks later in cognitive engagement exam two, the BE and 
SPS groups performed significantly better than the control group and both BE and SPS performed significantly better 
than GG. 

All three treatment groups performed within nine degrees of each other on the CE exam one. However, on 
CE exam two, SPS and BP performed significantly better with scores of 36.422 and 55.107 degrees respectively 
compared to the control group and GG with scores 141.394 and 133.335 degrees respectively. Results were 
analyzed similarly for the effect of the amount of time a participant took to comp lete the exams (time on task) based on 
group membership. Time on task (TT) for cognitive engagement (CE) exams one and two were used as the dependent 
variables and group membership (GM) and field articulation (PA) were used as between group factors. This analysis 
revealed a significant main effect for TT (P = 8.282, p < .05) and GM (P = 3.846, p < .05). Two-way interactions 
between TT and GM (P = 5.893, p < .05) were also found to be significant. No other tests were found to be 
significant. 

The main effect for TT and GM and the interaction effect of TT and GM were significant. Therefore, a 
Tukey One-Way ANOVA was performed to further analyze mean differences. When analyzing CE exam one, all 
treatment groups completed exam one significantly faster than the control group. Between group time performances 
were not significant. 

Results for cognitive engagement exam two showed only the GG treatment group performing significantly 
different compared to the control group - this time investing less time than the control group. Both BP and SPS 
invested significantly less time than GG. 

The cognitive engagement main effect and cognitive engagement and group membership interaction effects 
were significant. Purther analysis using a Tukey one-way ANOVA showed no treatment group performed 
significantly better. When participants performed their second GIS cognitive engagement exam two weeks later, 
significant differences were found in the BP and SPS models compared to the control group. Between the SPS and 
BP pedagogy, participants who received the SPS pedagogy performed best. When analyses were conducted between 
groups, BP and SPS performed significantly better than the GG group on the second GIS cognitive engagement exam 
with participants who received the SPS pedagogy once again performing best. 

When performance was compared to time on task (TT), all treatment groups took significantly more time 
than the control group when completing cognitive engagement exam one. No significant differences were found 
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between groups. When completing cognitive engagement exam two, the control group, BF and SPS group members 
took a significantly longer amount of time than the control group. 

Analyses of Participants’ GIS Procedural Knowledge 

GIS procedural knowledge refers to the ability to understand the basic procedural skills when using 
Arc View. Procedural knowledge was measured immediately after instruction and two weeks later. The procedural 
knowledge results were also analyzed using a repeated measures ANOVA. Procedural knowledge (PK) tests one and 
two were dependent variables with field articulation (FA) and group membership (GM) used as the between group 
factors. This analysis revealed significant main effects for procedural knowledge (F=71.985, p < .0001) and group 
membership (F=13.971, p < .0001) and a significant interaction effect between procedural knowledge and group 
membership (F=5.349, p < .05). No other tests were found to be significant. 

The main effect of procedural knowledge and group membership and the interaction effect of procedural 
knowledge and group membership were significant. Therefore, a Tukey One-Way ANOVA was performed to further 
analyze mean differences. All treatment groups performed significantly better than the control group. Between 
group analyses revealed that BF and SPS performed significantly better than GG. 

Participants’ performances on the second procedural knowledge exam revealed that BF and SPS performed 
significantly better than the control group. Between group analyses revealed that only BF performed significantly 
better than GG. 

The procedural knowledge and group membership main effects and the interaction effect between 
procedural knowledge and group membership were significant. Further analysis on procedural knowledge exam one 
showed all treatment groups performed significantly better than the control group and the BF and SPS groups 
performed significantly better than GG. Further analysis on procedural knowledge exam two revealed BF and SPS 
performing significantly better than the control group and only BF performing significantly better than GG on 
between group analysis. 

GIS Performance Qualitative Analyses 

Personal interviews were conducted that provided in-depth analyses of all GIS instructional methods. The 
personal interviews warranted a qualitative approach to data analysis because they helped identify advantages and 
disadvantages of the GIS instruction by recognizing emerging patterns across group membership. 

Data analyses were divided into two categories based on the questions that were posed during the personal 
interviews: 1) the issues that impacted GIS learning with each pedagogical model and 2) the extent preservice 
teachers believed they were able to integrate GIS within their future classrooms. 

The most common themes found in the interviews broken down by positive and negative responses. A 
positive response is defined by participant comments regarding the GIS pedagogical model they received during 
instruction as supportive to learning GIS. A negative response is defined by participant comments regarding the GIS 
pedagogical model they received during instruction as less than supportive to learning GIS. 

Control Group 

The common themes on issues that impacted GIS learning within the control group were all negative. The 
most common theme in the interviews was that participants were “completely lost” and at times “wanted to quit.” 

The participants understood the task, but didn’t know how to apply the skills they learned in the “Quick-Start 
Tutorial” to solving the GIS exam. Holly commented, “I followed the tutorial just fine, but when I had to use what I 
was supposed to learn in the tutorial in solving a problem with GIS, I had no idea where to start.” The second most 
common theme was the request for more guidance and training. Four of the five participants noted that one class 
period of learning GIS in this approach was not sufficient. More guidance and training is required if an instructor is 
going to use self-guided tutorials to have the students teach themselves GIS. Matt said, “I think that GIS would be 
great in the class, but after this experience, I am very hesitant to try to use this technology.” 

Basics First 

All BF pedagogy themes were positive. The first main theme was that access to an instructor to ask a 
question was a great benefit in learning GIS. In fact, all participants interviewed commented that they did not use the 
web-based module, but would rather wait to ask the instructor the question. Holly said, “I would much rather have a 
live person who I can ask questions than search on the web to find the answer. For me, it is much more efficient.” 
Having access to the instructor led to the second most common theme which was the opportunity to practice the GIS 
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procedural steps until they felt “comfortable.” Four of the five participants noted that being able to practice the 
procedural steps and being able to ask questions was an excellent way to learn. Martha said, “I was never stressed 
and I understood what I needed to do on the screen to perform the designated task.” 

Stmctured Problem Solving 

Four of the five common themes found in the interviews with the participants that received the SPS 
pedagogy were positive. The first positive theme, which all participants mentioned, was the opportunity to watch 
videos that reinforced learning at any time. Julie said, “I couldn’t believe how they helped! After I figured out how 
to switch between the web browser and ArcView, I had no problem learning the GIS. The videos were well done!” 
Anthony said, “If it wasn’t for the videos, I think I would have quit on the spot. There was no way I was going to be 
able to learn how to use the program without that assistance.” 

The second and third themes were also related to the videos. They included the opportunity to stop, 
rewind, and fast-forward the videos at any time and that the majority of the knowledge needed to perform well on the 
exams was answered within the web-based module. The opportunity to watch the videos was the most common, 
however, four of the five participants commented about the opportunity to manipulate the video to find the correct 
answer. Beau said, “I have to admit that I think I rewound the video at least three times. Granted, it took a bit of 
time, but it was worth it in the end.” Related to this were that the answers were readily found in the modules. Matt 
said, “It took me a while sometime to find the answer, but I was surprised that everything I needed was either in the 
text or in the videos.” 

Lastly, three to the five participants said the authentic practice questions were “excellent.” The participants 
liked having the practice GIS questions to learn ArcView and also learn how to apply the knowledge they had 
learned. Alan said, “When I did the practice questions I finally could apply the skills that I was learning. The 
questions were excellent in getting me to think about why I would use the specific skill in the GIS environment. The 
only drawback was that it took more time that I had wished.” Sheryl also said, “It all made sense to me when I started 
answering the practice questions. I could read and think about my task and then I could find the answer in the 
videos.” 

Even though the interview was dominated by positive remarks, the participants stated they would have liked 
to have had immediate feedback from an instructor during their learning experience. Cassie said, “The videos and the 
practice questions definitely helped me, but it would have been nice to ask an instructor questions at times.” 

Guided Generation 

Guided generation participants had a very different experience in their GIS learning. The only positive theme 
was the QuickTime video used for setting up the task to be completed. All participants thought it was an “excellent” 
way to “motivate” the learner and to “set the stage” for GIS learning. Chrissy said, “The video was excellent. I 
completely understood what we were supposed to do.” Bob said, “I had never seen anything like [the video]. I think 
it could be used in more than just learning GIS.” 

All other common themes were negative regarding the GG pedagogy. All participants said they felt either 
“lost” or “frustrated.” They all commented that they understood what they were supposed to do, but they didn’t 
know where to start. Even after the participants started to understand their task, four of them said that it took too 
much time to learn one skill. Joe said, “It seemed as if I spent fifteen minutes just to learn how to add one layer.” 

Once a skill was learned, the participants said they didn’t know how to “apply the skill.” Joe further commented, 
“Once I did learn how to import a layer, I really didn’t know what to do next. It seemed as if I was spending most of 
my time playing and really not getting anywhere.” 

The last questions of the interviews and one question on the demographic questionnaire pertained to how 
likely the participants would be to integrate GIS into their future classroom. 

Control Group 

Eifteen percent of the control group responded “maybe “ when asked if they feel they could integrate GIS 
within their future classroom. This statistic was supported in the interviews when all participants interviewed stated 
additional tutorials would be needed to learn GIS effectively. Eour of the five stated they would need much more time 
to learn ArcView before they would consider integrating GIS within their classroom. Marlene said, “I really was 
frustrated when only going through the tutorial without any other instruction. I felt like I was missing the entire point 
of doing the exercise.” Eour of the five participants also said they needed suggestions on effective integration. Matt 
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stated, “I actually think I would try to integrate GIS in my classroom, but I really need some ideas on effective 
integration and different types of lessons.” 

Basics First 

The BF group of participants felt very comfortable with GIS and 82% of them stated they would “definitely” 
integrate GIS within their future classroom. Roni said, “I really felt like I learned the concepts and use of GIS pretty 
well. My next big step is to practice and think about how I will use it in my class.” All participants interviewed said 
they would like a “little more practice,” but could definitely see the application of GIS technology. All participants 
commented that they relied heavily on the instructor and therefore, would really like some additional integration 
ideas. 

Stmctured Problem Solving 

Much like the BF group, 90% of the SPS participants said they would “definitely” integrate GIS within their 
future classroom. The participants stated that having access to the web-based modules for “further learning” and as 
a “refresher” would be excellent. Four of them asked if they could use the web-based modules with their future 
students. Jackie stated, “I definitely will try to integrate GIS in my future classroom. I would really like my students 
to have access to videos like we did. Would that be possible?” Three of the participants also said they would like 
integration ideas. Marlene said, “I would really like to see what other participantss have done with GIS. Some sample 
lessons or even videos like we had about GIS, but showing participants teaching would be nice.” 

Guided Generation 

Only 40% of the GG participants felt they would “maybe” integrate GIS in their future classroom All 
participants interviewed said they would need effective integration ideas for their classroom and more experience. 
Beth stated, “Not to be negative, but I really don’t see myself integrating a technology like this unless I have much 
more practice and experience. I need more stmcture and definitely more help!” Mark said, “I would like to use 
technology like this, but I really would need assistance in learning it better. I can’t imagine using it in my classroom 
when I have no idea of knowing what my students should be doing with it.” 

Personal interviews found participants who were in the BF and SPS treatment groups had an overall much 
better GIS learning experience than those in the control and GG groups. The BF group reported that access to the 
instructor was most beneficial while the SPS treatment group noted that access to the QuickTime videos proved most 
valuable. All participants who received instruction in the control and GG groups were very negative about the 
instruction they received stating they were “lost” during their learning experience and needed much more assistance. 

When asked about the future integration of GIS in their classrooms, the BF and SPS treatment groups were 
most positive and optimistic. The participants saw the relevance and need for GIS integration and were willing to try 
to integrate GIS within their future classroom. Many participants noted they would also appreciate the assistance 
from the web-based modules when they are teaching their students, participants in the control and GG groups were 
not willing to try to integrate GIS unless they would receive more opportunities to learn GIS so they could see the 
relevance of the technology. 

Field Articulation and Response Confidence 

Field articulation had no effect on participants performance on either exam. In regards to response 
confidence, significant Pearson correlations showed correlations between response confidence and participant 
performance on all exams. Therefore, as response confidence increased, so did performance showing participants 
were not guessing at their answers. 

Discussions and Recommendations 



GIS Performance Summary 

This study found participants who completed the first cognitive engagement exam did not perform 
significantly different depending on the GIS model of instruction they received. However, when participants returned 
two weeks later to complete the second cognitive engagement exam, participants in the BF and SPS treatment groups 
performed significantly better than the participants in the control group and in the GG treatment group. The mean 
scores were markedly different with the BF group scoring 86.2868 degrees and the SPS group scoring 79.5863 degrees 
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significantly closer than the control group to the correct answer. Likewise, the BF group scored 78.2281 degrees and 
the SPS group scored 71.5276 degrees significantly closer to the correct answer than the GG treatment group. 

Participant performance on the first GIS procedural knowledge exam revealed participants in the BF, SPS, 
and GG treatment groups performing significantly better than the control group, while the BF and SPS treatment 
groups performed significantly better than the GG treatment group. The second GIS procedural knowledge exam 
taken two week later revealed BF and SPS treatment groups performed significantly better than the control group and 
BF treatment group performed significantly better than the GG treatment group. 

The question that needs to be answered is why did the differences occur in the cognitive engagement and 
procedural knowledge exams? A closer look as to why they preformed the way they did was found in the personal 
interviews. 

Implications for Designing Learning Environments 

Based on the data from this study, the following consideration should be taken when developing GIS 
instruction for preservice teachers. They are I) instructors of GIS may have a tremendous influence on GIS learning 
using the “Basics First” pedagogical model, 2) GIS instruction that is not accompanied by an instructor within the 
room should have practice questions that set the GIS task within an authentic situation, 3) access to QuickTime 
videos for GIS skill understanding and review significantly enhances GIS learning, 4) GIS learners need instruction 
and direction in their learning, and 5) understanding the procedural knowledge of a GIS influences learning. 

The “Basics First” (BF) treatment group performed extremely well on all exams. They noted that access to 
the instructor was most important in their GIS learning. They would rather wait to ask the instructor than looking on 
the web-based module for the answer. Therefore, the quality of the GIS instructor was very important when learning 
GIS with this model. If instructors with different GIS backgrounds and understanding use the BF pedagogy, one 
must ask the question, “Will the GIS learners perform at such a high level?” 

The “Structured Problem Solving” (SPS) treatment group also performed significantly well with access only 
to the “structured practice questions” and QuickTime videos. Therefore, if one were going teach GIS online or would 
not be able to offer assistance at a level that is required in the BF treatment group, the SPS approach would be most 
beneficial. The success of the GIS learning environment following the SPS model relies heavily on the development 
of the QuickTime videos. All participants interviewed stressed how important these were for their learning and how 
the ability to stop, fast-forward, and rewind the video had a tremendous impact on their learning. The second most 
important feature related to the SPS model are the “structured practice questions.” The participants noted how 
beneficial it was for them to practice the GIS skills with “direct application.” The participants knew why they were 
doing the specific GIS skill. Therefore, practice questions where students are able to see the direct application of the 
question to real life can be very beneficial to GIS learning. 

The “Guided Generation” treatment group felt that they understood what they were supposed to do after 
they watched the authentic video, but didn’t “know where to start.” They stated that they needed to know how to 
apply the GIS skills to solving the task at hand. Therefore, a constructivist approach that provides no instructor 
guidance with GIS learning was not found to be effective and used with caution. 

Lastly, procedural knowledge and GIS performance significantly correlated on all exams. Therefore, when 
designing a GIS learning environment, GIS basic procedural skills should be taught and reinforced for quality future 
GIS performance. Participants within the GG treatment group received no instruction on procedural knowledge and 
performed poorly. Thus, even though setting up the environment with an authentic video may seem important, an 
understanding of GIS basic skills should be taught first. 
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Meeting State Mandates for Inservice Technology Integration Training 
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This paper details one university's approach to meeting state mandates for technology training for in- 
service teachers, and the results of a continuous, feedback -driven project to evaluate for technology 
integration through a educator electronic portfolio development process and field-based observations. 
Portfolios are assessed for national and state technology standards. Educators are also observed in their 
schools for the application standards in their classrooms, offices, and counseling centers. Assessment and 
evaluation results are discussed. 



Introduction 

Federal, state, and local agencies are investing billions of dollars to equip schools with the technology that 
may well be the key to improving the learning experience of our nation’s youth. Despite these gargantuan 
investments, only 20 percent of the 2.5 million teachers currently working in public schools feel comfortable using 
these technologies in the classroom (U.S. Department of Education, 1999; Technology Counts, 1997). As reeducating 
the existing teaching force to take full advantage of these technology tools will require expensive professional 
development over many years, much of the focus in recent years has been on ensuring the technology competency 
of new teachers. Initiatives such as the Preparing Tomorrow’s Teachers to Use Technology as well as the adoption 
of NCATE 2000 standards, which require teacher education programs to address technology preparedness, have 
gone a long way to prepare the estimated 2.2 million new teachers needed in the next decade (Milken Exchange on 
Education Technology, 1999). 

The focus is expanding, however, to address inservice teachers. Current literature indicates an increasing 
trend towards educational accountability, and in particular, teacher accountability. The No Child Left Behind Act of 
2001 (Public Law 107-110) reinforces states’ policies on improving education by calling for increased accountability 
and improved teacher quality. This act also supports staff development for teachers and administrators in the area of 
technology integration training and leadership development for technology selection and support. States receiving 
technology funds must spend 25 percent on training (Skinner, 2002). Some states have moved to requiring inservice 
preparation in technology integration as a requirement for teaching certificate renewal. Currently only four states 
require teachers to participate in technology staff development for certificate renewal in 2002: Connecticut, North 
Carolina, South Carolina, and Virginia (Blair, 2002). Others are moving in this direction. 

Georgia is one of those states with a plan in place to mandate in-service teacher technology training. While 
the state of Georgia had originally passed a law that all certified teachers would be required to document their 
technology competency for certificate renewal in 2001, the deadline was delayed until 2006 due to insufficient 
resources available to train the estimated 1 1 1,000 Georgia educators (Georgia Professional Standards Commission, 
2001). 

This paper details one university’s approach to meeting state mandates for technology training for inservice 
teachers. The following sections will review current professional literature on teacher accountability, technology 
skills for certification, and evaluation for teaching competencies, provide background on education mandates in the 
state of Georgia, provide an overview of project development scope and significance at Georgia State University to 
meet state mandates, and review assessment and evaluation data to date on the inservice teacher technology 
competency as well as developing issues in project continuance. 

Review of Literature 



National Trend for Teacher Accountability 

The national trend toward educational accountability continues. Two decades have past since the National 
Commission on Excellence in Education declared us a “nation at risk” due to the “rising tide of mediocrity” in our 
educational foundations (1983). In 1987, the Carnegie Task Eorce created the National Board for Professional 
Teaching Standards (NBPTS) in an effort to increase teacher quality and reward those who demonstrate excellence in 
teaching by awarding National Board Certification (National Board for Professional Teaching Standards, n.d.). At the 
same time, many professional educational organizations such as the National Council of Teachers of Mathematics 
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(NCTM) and American Association for the Advancement of Science (AAAS) were writing standards documents, 
creating challenging curriculum and calling for “state of the art teaching” (Zemelman, Daniels & Hyde, 1998, p. X). 
Continuing in the same vain. President Clinton and the 103”* Congress passed the Goals 2000: Education Act of 
America (1994). In this legislation. President Clinton called for effective uses of technology in teaching and learning 
that promote opportunities for students to meet the National Education Goals. Among other things, his presidential 
plans called for teachers to become technology literate. Paralleling these accountability and reform efforts, the 
International Society for Technology in Education (ISTE) began developing technology standards for students and 
teachers (National Educational Technology Standards for Teachers, 2002). Released in 2000, the third edition of the 
National Educational Technology Standards for Teachers (NETS-T) outlined six standards areas with performance 
indicators for applying technology in educational settings. Today, President Bush’s No Child Left Behind Act of 
2001 (Public Law 107-110) reinforces state’s policies on improving education by calling for increased accountability 
and improved teacher quality. This act also supports staff development for teachers and administrators in the area of 
technology integration training and leadership development for technology selection and support. States receiving 
federal technology funds must spend 25 percent on training (Skinner, 2002). 

Technology Skills for Certification 

Since the development of the NETS-T standards. Colleges of Education, state and local education agencies 
as well as accrediting bodies such as the National Accreditation for Teacher Education (NCATE) have begun 
requiring educators to demonstrate mastery of technology integration. Currently, 3 1 states and the District of 
Columbia have adopted, adapted, aligned or referenced the NETS for teachers in their state department of education 
documents which include certification and licensure requirements (Use of NETS by State). 

In 1997, NCATE concluded that a majority of teacher education programs were not doing what needed to be 
done in terms of preparing teachers to teach in the 2L* century classrooms. NCATE recommended that its 
accreditation body recognize technology education for teachers as central to the teacher preparation process. As a 
result, NCATE raised the bar. The newest NCATE unit standards now require teacher candidates to be able to 
“appropriately and effectively integrate technology and information literacy in instruction to support student 
learning” (2000, p. 8). Presently 26 states and the District of Columbia require technology training or coursework for 
initial teacher licensure and 7 states require an assessment of technology competence before granting initial 
certification (Blair, 2002). With the newest NCATE standards in place, more states will begin to make similar 
requirements of their college graduates. 

Currently only 4 states require teachers to participate in technology staff development for certificate renewal 
in 2002: Connecticut, North Carolina, South Carolina, and Virginia (Blair, 2002). Others are moving in this direction. 
Eor example, this fall Texas’ State Board for Educator Certification (SBEC) approved educator certification standards 
in Technology Applications for their beginning teachers (n.d.). Beginning fall 2003 Educators seeking initial licensing 
are required to pass a Pedagogy and Professional Responsibilities test embedded with the state’s new technology 
standards. The SBEC recommends that all educators should strive to meet State Board of Educator Certification 
standards required for all beginning educators. In addition, they have created a Master Teacher Technology 
Certificate program to provide select teachers with additional technology training in order to assist their colleagues 
and students with technology integration. In October 2003 teacher certification applicants in Illinois will take a new 
Assessment of Professional Teaching which includes a section on core Technology Standards (Illinois State Board 
of Education, n.d.). Although veteran teachers will maintain their current endorsements, the Illinois State Board of 
Education states that the core Technology Standards will be used to guide the professional development of teachers 
and administrators. In Georgia, in-service teachers and administrators are required to complete either the Georgia 
Eramework for Integrating Technology in the Student-Centered classroom (InTech) staff development program or a 
state approved technology integration course prior to renewing their teaching certificate. Portfolio assessments are 
used to document mastery of state technology standards. All educators are expected to meet this requirement 
beginning in 2006. 

Evaluation for Technology Competencies 

Assessments are central to educational reform and accountability. As a favored strategy by lawmakers and 
government agencies, their intended purpose is to “promote higher standards of teaching, more powerful learning, 
and more credible forms of public accountability” (Hargreaves, Earl & Schmidt, 2002, p. 70). When it comes to 
evaluating technology skills, three types of assessments are common: skills -based self-assessments, electronic 
portfolios, and observation instruments. Alternative to pencil-and-paper tests, these performance assessments are 
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intended to motivate teachers to take more responsibility for their own learning, to utilize complex problem-solving 
skills, and to embed learning in authentic activities which promote a synthesis of information and skills (Herman, 

1998; Hargreaves, et. ah, 2002). 

Technology skiU-based self-assessment surveys seek to measure a teacher’s “level of skill, literacy, 
integration and attitude towards technology and learning’’ (Anderson, 2000, Formal Assessments section, para. 1). 
National surveys have been conducted by several researchers in an attempt to determine characteristics of exemplary 
technology users, belief, practices and types of computer use by teacher leaders and patterns of technology 
integration (Hadley & Sheingold, 93; Becker, 94; Riel & Becker, 2000). State agencies, school districts and local 
schools have created or modified existing surveys. These tend to be either short rubric-based or Likert scale 
instruments. Assessments include those such as the Levels of Technology Implementation (LoTi) scale (Moersch, 
1995) and the Mankato Survey of Professional Technology Use, Ability and Accessibility (Mankato Public Schools, 
1997). The purpose of these instruments is for staff self-evaluation and decision-making about technology staff 
development. 

Electronic portfolios are becoming a common means of validating both pre -service and in-service teacher 
performance (Lyons, 1999; Kariuki & Turner, 2001; Wright, Stallworth, & Ray, 2002). These alternative assessments 
provide learners opportunities to reflect on their practice, evaluate their own learning and progress towards clearly 
defined goals (Corbett-Perez & Dorman, 1999). Electronic portfolios not only encourage teachers to demonstrate 
pedagogical competencies but also foster continued growth in their technology skills. The results of such efforts 
may directly impact students. Teachers who create electronic portfolios are more likely to integrate technology in 
their own classrooms (McKinney, 1998). 

Observation of technology integration competencies is another means to evaluate the level to which a 
teacher can meet standards. Researchers conduct qualitative observations to examine teachers’ beliefs about 
technology and technology integration practices (Ertmer, Addison, Lane, Ross & Woods, 1999; Dias, 2000; Pierson, 
2001). Collecting data from classroom observations of technology use is imperative, since the effectiveness of 
teaching practice is enmeshed in the myriad details of maintaining student engagement. As summarized by Painter 
(2001), “Successful integration of technology requires not only knowledge of the technology and its potential use 
but also the skill to plan and execute a good lesson (of which the technology is only a part)’’ (p. 23). Pew observation 
instruments have been developed and decimated that target teaching with technology. 

Background: State of Georgia Mandates 

The state of Georgia has had several state initiatives to increase student use of technology and likewise to 
increase teacher technology competency. These initiatives include the use of lottery funds to fund technology 
initiatives in the schools, state funding for technology training centers, and a state legislation. House Bill 1187, which 
increased funding for technology specialist positions in schools. (Technology Counts, 2001). H.B. 1187 als o 
mandated pre -service and in-service technology competency as of June 1, 2001. These initiatives are not without 
limits, however. Details about each of these initiatives as well as infrastmcture limitations that limit Georgia’s means 
of meeting these mandates are detailed below. 

The Georgia Lottery was created in November 1992 by the people of Georgia to enhance education funding 
(Georgia Lottery Corporation, 2002). Lottery proceeds fund a variety of educational initiatives, including tuition 
grants, scholarships, and loans to undergraduate students and teachers who seek advance degrees in critical areas of 
need; voluntary prekindergarten programs; and technology grants to train teachers in the use and application of 
advanced technology and capital outlay projects for educational facilities (hardware, software, networking, teacher 
training centers, surveillance cameras and other safety hardware) (Georgia Lottery Corporation, 2002). Since 1992, the 
lottery has contributed more than $5 billion to education in Georgia. Of this, more than 1.8 billion as been 
appropriated to capital outlay projects and technology grants for schools. Lottery funds are meant to supplement, 
not supplant, traditional educational funding. In fiscal year 1998, Georgia funded $55.5 million in technology and 
capital outlay projects, $52.9 million of which came from lottery funds (Technology Counts, 1999). Estimated 
technology funding for fiscal year 2001 was $65 million (Technology Counts, 2001). A portion of state technology 
funding goes to the operation of thirteen technology training centers. The main program offered at these centers, 
InTech, is a staff development program for educators that focuses on integrating technology into the curriculum and 
classroom instruction. 

Early in 1999, Georgia Governor Roy Barnes formed an Education Reform Study Commission to look at ways 
to improve Georgia’s schools. Governor Barnes used the results of the commission’s study to produce the A Plus 



67 




Education Reform Act of 2000 (2002). Out of the act came technology-related initiatives that impact practicing 
educators. First, the act funded one technology specialist position for every 1,100 students. Previously, one 
technology specialist position was funded for every four schools (Technology Counts, 2001). Second, the act 
mandated in-service educator technology integration competency: 

Legal Reference: Code Section 20-2-200; Subsection (b)(2) 

(b)(2) Before granting a renewable [teaching] certificate to an applicant, the commission is authorized to 
require the applicant to demonstrate either satisfactory proficiency on a test of computer skill competency or 
successful completion of the phase one InTech training model at a state educational technology center or by a State 
Board of Education approved redelivery team shall be acceptable for certificate renewal purposes. 

The A-i- Education Reform Act in the state of Georgia requires that all certified educators in Georgia 
demonstrate competence in the application of instructional technology as outlined in the Georgia State Technology 
Standards for Teachers. The Special Georgia Technology Requirement is grounded in the premise that all educators 
must become competent in the use of technology to attain high levels of student achievement. To this end, the law 
stipulates that the state funded technology training program, InTech, which is offered at the thirteen state funded 
technology training centers, be the defacto method for educators to achieve technology competency for 
recertification. The mandate was to go into effect for certificate renewals as of June 1, 2001. 

However, the state soon realized the implications of the law and the reliance on state technology training 
centers. The law, as written, indicated that within a period of five years, the thirteen state technology training centers 
would need to deliver the 50-hour InTech program to over 1 10,000 educators. This far exceeded the capacity of the 
state technology training centers. To alleviate this overburdening of state resources, that state enacted a delay in the 
application of the law to June 1, 2006. Also, the legislature charged the Georgia Professional Standards Commission 
(PSC), the agency that manages educator certification, with two tasks: to develop a plan for alternative pathways for 
educators to meet the state requirement, and to formally approve and monitor these alternative pathways developed 
by state and other local agencies, school districts, private schools, and colleges and universities. Therefore, multiple 
agencies share in the responsibility for ensuring that all educators are competent in integrating instructional 
technology. 

State Requirements and Program Development 

The Georgia Professional Standards Commission (PSC) set forth several stipulations that must be included 
in any proposal submitted for a course or professional development program to be considered as an approved 
alternative pathway to meet the state technology requirement. First, any course or program developed and proposed 
must address multiple audiences, including those in instructional, support, service, and administrative roles, as well 
as educators with special learning and assessment needs. Second, any application for PSC approval must include a 
matrix demonstrating how the course or program maps and correlates with state technology standards. Third, the PSC 
mandated three format stipulations: programs offering professional development and assessment must also offer 
educators the opportunity to “test-out” of the requirement; approved programs must make use of an electronic 
portfolio for assessment purposes; and approved programs must include an on-site, field -based observation of each 
applicant. The course/program application packet can be downloaded from the PSC at: 
http://www.gapsc.com/TeacherEducation/Documents/Tech_App.pdf 

Course in place: IT 7360 Technology for Educators 

At Georgia State University (GSU), a resource-based learning environment was developed in fall 1998 to 
serve as the basis of an elective course in technology integration methods for graduate education students. The 
course, IT 7360, Technology for Educators and its related resource laden WWW site, incorporates a problem- 
centered, activity-based approach where the computer applications are anchored in authentic and familiar contexts in 
which teaching and learning occurs (Cognition and Technology Group at Vanderbilt, 1991; Vygotsky, 1978). This 
approach is based on the view of an open learning environment in which learners have direct input on the direction 
of the course based on their needs (Hannafin, 1999; Hannafin, Hall, Land, & Hill, 1994). In navigating through the 
environment and tackling challenges, it is proposed that educator-students will also develop self-directed learning 
skills, which will serve them well as they professionally develop as educators. Along with confidence in using the 
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technology, self-directed learning skills have been identified as a characteristic of successful technology-using 
teachers (Shoffner, 1996). The RBLE can be accessed at http://msit.gsu.edu/IT/Teachers. 

In addition to introducing and reinforcing technology integration skills, the course reviews teaching and 
planning methods for the technology-enhanced learning environment. Educator-students review instmctional 
objectives, national, state, and local curriculum standards, lesson planning, evaluation, and assessment. In the 
Technology for Educators course at GSU, the technology is immersed in learning about what being a educator entails 
- briefly, planning, learning theory, instructional strategies, classroom management, and assessment. Throughout the 
course, educator-students demonstrate their technology integration skills in a variety of activities which focus 
simultaneously on both what they can do with the technology, personally, and their ability to plan for their students 
to meet curriculum requirements while making use of a variety of technologies. The course makes use of multiple 
methods of assessment, including Technology Integration Planning and Skills samples, a paper-based portfolio, rich 
with student and educator technology artifacts, and a series of reflection papers on teaching with technology. 

Audiences Served 

While the course at GSU, like most other technology integration courses taught at the university-level, 
focused on what the classroom, Georgia’s H.B. 1187 specifically called for the all certified educators to meet the state 
technology standards. In the call for proposals for programs to meet the Special Georgia Technology Requirement, 
pathways for variety school- and district-level educators had to be included. Specifically, any program approved by 
the Professional Standards Commission must provide suitable professional development and assessment for those in 
instructional and instructional support positions (classroom teachers, lead teachers, school library media specialists, 
teacher support specialists, and guidance counselors), as well as those in administrative or leadership positions 
(principals, assistant principals, district administration, as well as the often overlooked specialized certification fields 
such as school nutrition directors and school psychologists). A final requirement indicated that all approved 
programs must provide accommodations for the professional development and assessment of those educators with 
special needs, as defined by the Americans with Disabilities Act. 

To determine if the course at GSU met the needs of both leadership and leadership populations, course 
objectives and assessments were mapped against the various Georgia technology standards. The mapping process 
and resulting adaptations are described in the following section. It was determined that the existing course would 
meet the special needs population requirements, as all programs and courses offered at GSU must provide 
accessibility and reasonable accommodations for persons with disabilities through the Office of Disability Services. 

Documentation: Alignment with Standards 

IT 7360, Technology for Educators, was designed to meet the earlier versions of the International Society for 
Technology in Education (ISTE) standards last distributed in 1998, and updated to match the 21 standards in five 
overarching categories of the National Technology Standards for Teachers. With H.B. 1 187 and the ensuing charge 
of the PSC, Georgia Technology Standards for Educators were developed and were included with the application 
materials to be downloaded from the PSC web site. Georgia State Technology Standards for All Teachers can be 
downloaded from the PSC at http://www.gapsc.com/TeacherEducation/Documents/TechStandards.pdf 

Each program seeking approval from the PSC to offer state technology training was required to document 
that each and every Georgia Technology Standard for Educators, Professional Development Objective, and 
Performance Indicators was met. State and National standards were compared for parity. As the Georgia State 
Technology Standards for All Teachers were based on the NETS-T, parity was established. The application packet 
included a matrix to be completed. The matrix contained six columns, three of which were contained the Georgia 
Technology Standards for Educators, Georgia Technology Professional Development Objectives, and Georgia 
Performance Descriptors for teachers (instmctional), service personnel, and administrators (leadership). Next, state 
technology standards were mapped against course objectives, sub-objectives, performance indicators, and 
assessment instruments. It was determined that all state performance indicators that addressed technology 
integration and planning skills for those in instructional positions were met in the course assignments and the 
culminating project, the portfolio, of the existing course. Equivalent course assignments and projects were created for 
those in administrative positions to meet the technology integration and planning skills performance indicators of the 
Georgia State Technology Standards for Administrators. It was also determined that all state technology application 
and management performance indicators could be assessed in the required field -based observation of the educator. 

Rubrics were used to document each educator’s performance. In addition to rubrics already in place to 
assess technology integration and planning skills, observation rubrics were created to assess application and 
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management skills for those in instructional positions (teachers and media specialists). A second set of rubrics were 
developed to measure technology integration, planning, leadership, and dissemination skills for those in local school 
and district administrative positions. Technology integration and planning rubrics are available for viewing on the 
course website at http://msit.gsu.edu/IT/Teachers/Assignments/3210/Samples/Rubrics/rubricmain.html. 

Project Development Requirements 

The PSC mandated three format stipulations that must be included in any proposal submitted for a course or 
professional development program to be considered as an approved alternative pathway to meet the state 
technology requirement: programs offering professional development and assessment must also offer educators the 
opportunity to “test-out” of the requirement; approved programs must make use of an electronic portfolio for 
assessment purposes; and approved programs must include an on-site, field-based observation of each applicant. 

Educators who had developed technology integration knowledge and skills must have the option of 
“testing-out” of any approved program. For IT 7360 and the related on-line resource based learning environment, 
meeting the required test-out criteria was not difficult. Course developers had always intended for the course to 
someday be delivered at a distance. The course was designed so, if needed, educators may progress through 
instmction on particular software skills and applications in an individualized, self-directed manner. The on-line 
learning environment for IT 7360 contains all necessary course materials for educators to learning and master skills, 
including software help sheets (tutorial directions) and recommended readings. Previous experience with course test- 
out options made the program developers cautious, however, resulting in a GSU-specified screening process to 
ensure test-out participant skills for success, as well as a timeline for the application and completion of the test-out 
option each semester. Educators electing to participate in the test-out option are still required to make application to 
the university and pay tuition. Educators who successfully complete the test-out option receive three graduate 
credits. 

A second PSC stipulation was for the inclusion of an electronic portfolio. To meet this requirement, GSU 
converted the culminating paper-based portfolio to a web-based format. This change precipitated a change in course 
content, as instructional time in developing websites (as opposed to single webpages) was added to the course. 

A final PSC stipulation was that all candidates must complete an on-site observation that demonstrates their 
abilities to integrate technology in accordance with their Job responsibilities. This stipulation goes beyond the 
assessment requirements of the state technology professional development program, InTech, which has no 
observation requirement. Two observation rubrics were developed to measure achievement of performance based 
Georgia Technology Standards for both instructional and support roles as well as for individuals with leadership 
certificates. The content validity of the observation assessment instruments were checked through the inspection of 
items and sample performance descriptors by experts in the area student teaching as recommended in psychometric 
texts (see for example Gay, 1992; Isaac & Michael, 1981; Kerlinger, 1992). Experts were asked to check for overall 
content and accuracy. Based on feedback from these experts, specific changes in content and wording were made to 
the instruments. 



Assessment and Evaluation 



Assessment 

Not all students who enroll in the GSU course use it to meet the Special Georgia Technology Requirement, 
as some have already met the requirement through the state InTech program, or are exempt due to National Board 
Certification. In the spring 2002 semester, of those who indicated they would use the IT 7360 course to meet the state 
requirement, 16 educators earned a grade of “B” or better in all technology integration and planning skills 
performance indicators (100 %). Field-based observations took place in a variety of instructional settings: elementary, 
middle, secondary, staff development, higher education, and public libraries were used as environments. Large group, 
small-group, individual, classroom, and laboratory instructional configurations were used. Technology resources 
included as little as one computer in a classroom, and as many as 38 computers in a laboratory setting, including 
many permutations in between. Content taught included mathematics, social studies, science, language arts, writing, 
reading, pre-reading, remedial, ESOL, computer science, and teacher staff development. Technology tools included 
spreadsheets, databases, graphics, writing, concept-mapping, word processing, desktop publishing, desktop 
presentation, internet research, creation of web pages, and use of weblogs for journaling. The purpose for educator 
technology integration in many cases was student productivity, but many educators also cited student motivation, 
the need to meet multiple learning styles, and enrichment. Fall 2002 semester observations are currently ongoing, but 
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to date demonstrate the richness of topics and applications present in the spring semester. An additional 3 1 
educators are participating this fall semester. To date, the educators represent a total of 16 county/school systems, 
and 55 different schools, both public and private. 

Evaluation 

During this first year as a pathway to meeting the Special Georgia Technology Requirement, several issues 
have surfaced in the strategies, management, and delivery of the IT 7360 course. These issues generally fall into one 
of two course requirements: the e-portfolio, and the field -based observation. 

Issues that have arisen with the e-portfolio can be grouped into the areas of access and skills. Regarding 
access, educators still do not regularly have access to webspace at their place of employment, and must use the 
webspace provided to all GSU students or a free webspace such as Geocities. One participant even used a “weblog” 
site. University student webspace has been moved more than once in the past year, and teaching the file transfer 
process continues to take a great deal of instructional time. 

This lack of regular access has also led to skills issues. Educators have little difficulty creating a single 
webpage, but struggle with design and file structure knowledge when asked to create an entire website, as they must 
to meet the e-portfolio requirement. To alleviate the design struggles, instructors created a template for educator- 
students to use in developing their e-portfolio. Educator file structure knowledge was not as easily solved. Strategies 
used to date include having educator-students first map their own e-portfolio website based on the requirements 
(objectives) using Inspiration®, and then creating templates as a class, modeling the linkage structure (instead of 
simply supplying the templates for use). After developing the personal templates, educator-students are also 
provided with the instmctor created templates. The process of developing their own templates assists learners in 
developing file structure knowledge allows them to relax and focus on their content. A recent skill issue has been 
relative linkages used in recently released web development software upgrades making web page editing difficult. 
Instructional strategies to address relative linkages knowledge are currently under development. 

Evaluation results of the on-site observation requirement are largely of a management nature. GSU, located 
in downtown Atlanta, is centrally located in the most populated area of the state. The university regularly draws 
teacher-educators from as distant as a 100-mile radius of the city. The scheduling, management, record keeping, and 
travel to and from on-site observations have turned into a time-consuming and expensive operation that has placed a 
strain on GSU resources. Initially, conducting on-site evaluations was not considered as part of faculty teaching 
load. The distances covered to conduct on-site observations has made the training of evaluators to ensure inter-rater 
reliability difficult, resulting in only two trained evaluators to date, further straining GSU resources. 

With increasing enrollments, however, management issues in scheduling and conducting on-site 
observations are being addressed. In fall 2002 semester, one faculty course load was awarded for every 20 on-site 
observations scheduled. Eurthermore, it is anticipated that two additional evaluators will be trained prior to spring 
2003 semester. As more educators use the GSU course to meet the technology requirement, more observations will be 
scheduled, and location groupings of observations may be possible, thus decreasing travel expenses. 

Conclusions 

The national trend toward educational accountability will undoubtedly continue. It is anticipated that more 
states will follow the lead of Georgia, Connecticut, North Carolina, South Carolina, and Virginia, and will mandate 
educator technology integration professional development and put competency gateways into place. The curriculum 
mapping versus standards methods as well as the strategies for meeting state mandates in this paper will hopefully 
assist and encourage other university programs in being proactive to meeting this large and important in-service 
teacher professional development need. 
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Augmenting Classroom Instruction with WebCT 

Ron Payne 

Oklahoma State University 

Less than ten years ago, if you mentioned the word “classroom” to any person on the street, you were apt 
to receive some form of subject recognition. To most of us, a classroom basically involved the actions of a teacher or 
some hardware that could be suspended, attached, or rolled into the room. If there were connections to the world 
outside the classroom, they usually involved some form of television, such as Channel One, close-circuit video, or 
some local station. Most of the other items usually associated with the classroom, like the chalkboard or the 
overhead projector, were largely considered standard for any general schoolroom and, in fact, a part of the general 
definition. The point is when we spoke of learning or the act of teaching in connection with a classroom, we rarely 
had to define the tools that had become associated with the norm. (Ko, Rossen, 2001) 

However, in those same ten years, we have begun to undergo a tremendous change. Whether the change is 
to be considered revolutionary remains to be seen; but we actually have a chance to be a part of what could be one 
of the greatest changes to the act of teaching in 2,000 years! To be sure, at one time or another, we thought the 
overhead projector, the television medium, and even the motion picture would change education dramatically. To 
some extent, they probably had some good effects, yet overall, the definition of and the act of teaching remained 
fairly constant. Students still sit in rows with the teacher at the front of the room. To use any new devices, we often 
had to endure scheduling conflicts, a considerable amount of expense, and some effort to work through the 
associated “entertainment” value. Often, by the time we had obtained the items, using them properly had to slip 
somewhat. Moreover, when the big day arrived that we were to use the media, there was no assurance that there 
would not be a fire drill or a mass absence for some type of event! 

Now, however, there is a tool available that requires no scheduling, little in the way of funds, especially 
when compared to setting up a video studio, and the tool is already perceived as a productivity tool more than one of 
entertainment. It is the computer, of course, but even more than that, it is the special software now available for the 
teacher. The software, or course tool, was designed by a teacher and is often used for distance education. But 
mostly, its real value is in the way it can support classroom instruction. And depending upon the involvement with 
such details a teacher wants to be engaged, (s)he can change the design, colors, or any other design elements, and 
(s)he can add instructional elements or take away instructional elements as needed. The teacher can administer a test 
or survey, show a movie, provide for an online discussion, use class e-mail, conduct an online chat or maintain a 
virtual office hour, and/or provide an audio effect to support realism or add narration when needed. But, even more, 
the course tool technology gives us a tremendous opportunity to change the very premise of what we define as 
classroom instruction. In no small way, it helps to make the possibilities of a constructivist’s environment a reality. 

Though the literature has suggested it for years, we are quickly approaching the time when we really are 
going to change our basic method of instruction from essentially teacher-centered activities to those directed by the 
learner. (Murphy, Cathcart, Kodali, 1997) And we teachers are beginning to see ourselves as “learning facilitators” 
instead of the traditional “know-it-all.” 

By using course tools, such as WebCT, we can even have more time to devote to answering students’ 
questions, providing uniquely individual instruction, and or making available additional tutorials for students having 
difficulties or wanting to go deeper into the course content. (Shih and Cifuentes, 2001) In fact, with a little practice, 
we can turn any classroom into a haven for personalized instruction as envisioned by James Keefe and John 
Jenkins(2002). 

In a very real sense, we can add the major concept of personalization to the traditional classroom: students 
setting their own pace of learning, learning materials based upon learning styles. With a tool like WebCT, we can 
begin our conversion to the future now. We can make course content available when our students are ready for it. 
We can selectively release content based upon test scores, students’ names, or just dates. 

In addition, a tool like WebCT can help us help our students gain academic confidence. Far too often, we 
have had students who suffered because they did not trust their efforts to produce “what the instructor wants.” As 
we attempt to provide an environment that supports our students’ attempts to construct knowledge, our job should 
be to help our students learn how to learn for those times when they must successfully solve their own problems. If 
we do, they will be able to engage in what could be a wonderful world where technology will help them attain the 
“best” learning experience that they define for themselves. (Adler and Rae, 2002) In fact, some suggest that we 
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should help students learn to set their own achievement goals and help them maintain their work habits by letting 
them see their progress. (Salpeter 2002) With tools like WebCT, we can show them what they are accomplishing and 
provide the extra experiences that once was available only “outside” the classroom and, therefore, rarely. The 
students can select the extra experiences and content they want while still being on hand for the core of the course. 

What about the importance of collaboration? Several authors suggest that we have spent far too much time 
making education a competitive enterprise. We need to make learning more of a team effort. (Heinich, Moldenda, 
Russell, and Smaldion, 2002) (Newby, Stepich, Lehman, and Russell, 2000) (Jenkins, J. and Keefe, J., 2002a) In 
WebCT, we can make teams easily. Students can exchange document and spreadsheet files, Web pages, and audio 
and video files. The instructor can randomly generate teams or assign students to teams. When the teams are 
generated, special discussion boards are set up for only the team members (and the instructor). Students can share 
ideas, suggest work strategies, and assign each other deadlines. They can begin to take some responsibility for their 
own learning. For many students, class stops being something to endure or merely to be a place to exhibit a passive- 
resistance to learning and become instead a place of active learning! 

In addition to providing online discussions for groups, WebCT provides for online discussions for the 
whole class. Interestingly, students’ additions to discussions tend to be well thought out, not the “off the cuff’ 
comments we may hear in the classroom. In addition, online discussions can be an effective means for disallowing 
certain diversity issues from adversely affecting a learning opportunity. Since the teacher can configure the 
discussions to be anonymous, students do not know whose contributions they read. Also, students discussing an 
issue online can be amazed to find that their thoughts or opinions are the same as those held by someone with a 
different societal background or ethnicity. In essence, the negative effects produced by stereotyping can 
successfully be combated by this technique, helping the school achieve one of the goals of ethnic diversity. (Banks, 
etal, 2001, p. 200) 

For a believer of the constructivist methodology, we can view the use of Web-enhanced classrooms as a 
very real attempt to gain the extra dialogues between our students and the content from which they are to construct 
their knowledge. We see three modes of dialogue and communication: 1) a dialogue between an instructor and a 
student; 2) the dialogue between and among students (learning from each other); and 3) a dialogue between the 
student and the instructional materials and resources. (Boettcher and Cartwright, 1997) And it seems that this 
method is very effective. Students do tend to learn more effectively when they interact with and process information 
than when they just hear about it. (Floffman 2001) Indeed, we seem to have a wonderful opportunity to move beyond 
the learning of “inert” facts to generating better mental models (Dede 1995) supporting their construction of 
knowledge and problem solving strategies. This may sound like a lot and it is. But with a tool like WebCT, so much 
of the course operation is standardized and even easy that the teacher is left free to concentrate upon important 
pedagogical issues. Students do not have to take a course on taking a course. The course tool tends to be intuitive. 
Like the faculty, due to the inherent standardized look and “feel,” students can devote their attention to the course 
content and the online experience. (Baker, 2001) 

And, for the utilitarian purpose of producing effective citizens for our world of tomorrow, we should use the 
technologies now available to us to help our students get used to working with them. They are already entertained 
by it. Now, let’s show them how to work effectively with them. Or, as one author suggested, “Given today’s 
increasingly technological world, educational institutions must produce students who are able to function 
comfortably in this world.” (Levine 2002, p.l8) 

But how? We may need a real example to see how some of the instructional “wishes” and “hopes” can be 
employed successfully. A quick survey of the constructivist’s literature can provide several concrete suggestions 
for interpreting theory into the “classroom.” An interesting case is provided by Flonebein (1996), who suggests 
seven goals for the design of constructivist learning environments. Flis list of the seven goals builds upon the work 
of Cunningham, Duffy, and Knuth (1993). Flonebein’ s goals follow: 

1. Provide experience with knowledge construction process. 

2. Provide experience in multiple perspectives. 

3. Embed learning in realistic and relevant contexts. 

4. Encourage ownership and voice in the learning process. 

5. Embed learning in social experience. 

6. Encourage the use of multiple modes of representation. 

7. Encourage self-awareness of the knowledge construction process. (11-12) 
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When faced with the “trouble” of translating the seven goals into action, one is reminded of Wilson’s (1996) 
plea for teacher to remain “vigilant to ensure that an environment includes proper support and guidance and rich 
resources and tools.” (5) It is in applying the goals that one does more than pay lip service to her/his beliefs about 
learning and teaching and takes action to help students attain the desired learning. An example of one instmctor’s 
attempts to apply the seven goals using WebCT to augment classroom instruction follows. 

1. Provide experience with the knowledge construction process. Following a presentation about the Internet 
and some of its educational uses, the students were given live instruction about HTML. Their tool during this 
session was Notepad, a small text editing package that is a resident of Windows operating systems. The students 
created three pages that linked to each other and included differing font colors and at least one picture. (Some 
students placed more than one picture.) During the next class session, the students were introduced to FrontPage. 
After a couple of sessions of instruction and practice with the software, they were assigned the task of creating their 
own Website. The purpose of the site was to introduce themselves, tell about their interests, display their pictures 
(they had taken digital images of each other earlier in the session), and provide links to their favorite Websites. By 
using the tool of their choice, no one selected Notepad, they met and solved the various challenges of producing a 
site. 

2. Provide experience in and appreciation for multiple perspectives. After the students uploaded their Web 
pages to the protected course site, they could view each other’s pages. They checked the links and looked at how 
others met and solved production problems that they had experienced in their own production. In addition, they 
noted the creativity displayed by each Web page and in some cases, worked to figure out how certain features had 
been accomplished. 

3. Embed learning in realistic and relevant contexts. Believing that every educator needs to “know their own 
mind” or view of typical educational issues in the event they are ever called upon to present their views to members 
of the community, the students are asked to write a position paper on one of four issues facing education. The 
issues are set forth as follows: 

Homework? Some people suggest children have too much, do they? For additional information, you might begin 
here. 

Socializing with friends ? Reports are school performance falls as time spent with friends increases? For additional 
information, consider this information. 

Extracurricular activities? Are sports too important? Check the additional information. 

Television? Is TV viewing a problem for students? For additional information, see this! 

Part-time jobs ? Do students work too much and too long? For additional information, see Gannon University's study 
hints page. 

In addition, the students are told to produce a PowerPoint presentation that they can use to present their position. 
Though they are asked to present their position, they are cautioned that, as potential educators, they should provide 
an “informed” opinion citing the sources provided as links and an additional one of their own. The authentic activity 
for this aspect of the course is for them to determine their own stand. 

4. Encourage ownership and voice in the learning process. During the second week of the course, after all 
the introductory exercises and learning to use the course Website are satisfied, the students are asked to provide a 
list of ten learning objectives they hold for themselves during their time in the course. Those objectives are 
combined, provided with appropriate links to course materials that will further them on their way to achieving the 
objectives, and posted in a prominent place in the course Website. 

5. Embed learning in social experience. Later in the course, the students are introduced to the concept and 
steps involved in performing a WebQuest. They are told by way of introduction that the WebQuest will help them 
move from Web Surfers to Web Searchers. They will learn to search with a purpose, not aimlessly wander through 
cyber space. To perform the WebQuest, they are divided into four research teams. They are then told to physically 
move to one of four locations in the room whereby they take part in a short team-building exercise. Then, they are 
introduced to the challenge requiring the WebQuest. Since most of these students will probably be in charge of their 
school’s Website by virtue of their having this course on their transcript, they are charged with the responsibility to 
perform as a team to determine five major rules that determine an exceptional educational Website. Then, they are 
assigned the task of finding five educational Websites that display each of their rules and they are to find five 
educational Websites that fail each of the five “tests.” They are provided a site that lists all or most of the 



76 




educational sites in Oklahoma that they are to study. Finally, they are given the task of reporting the findings of their 
research via a group Website inside the course Website. In the beginning of the three-day exercise, the instructor 
moves from group to group to handle any special problems, answer non-routine questions, or to provide appropriate 
guidance. By the second and third day, the instructor “fades” away and allows the groups to function as 
independent teams that assigns tasks for each member and sets appropriate deadlines. After the completion of the 
quest, the students engage in a special essay “examination” during which they reflect and write about their 
experiences, recount their activities, and, hence, their learning. 

6. Encourage the use of multiple modes of representation. In several cases through the time during which 
various pieces of relevant software or certain activities are presented, the students are presented at least two modes 
of gaining information. First, the traditional voice, picture, “doing” type of instruction is presented during class. On 
the course Website, there are additional “presentations” that augment the in-class work. The presentations may be a 
simple Web tutorial, as in the case of learning FrontPage. To enrich the in-class presentation of Boolean logic for 
Web research, a couple of Viewlets© are presented to help the students quickly “cement” their new search power. 

7. Encourage self-awareness of the knowledge construction process. The discussion tool is ideal for the 
metacognitive effort. In responding the questions about the topic of study posted by the instructor, students are 
asked to respond to both the instructor’s questions and to the questions and/or comments of at least five of their 
classmates. In addition, when the instructor finds a need to add a comment, he has found a unique way of further 
enriching the discussion tool by linking to a video inside the message. He says something similar to, “ ... for a 
report dealing with MIT’s work on artificial intelligence, click here.” When the student clicks the link marked “here,” 
a short video plays showing the work of several MIT researchers as they “teach” machines to respond as humans. 
Predictably, the video brings about additional discussions about humanists versus machines and, then, often leaps 
into what the students can expect in the future! 

In conclusion, WebCT can assist the constructivist build a meaningful environment for students. In so 
doing, the practicing teacher risks taking her/his classroom out of the “safe” zone of centuries-old tradition and 
leaping deep into the future. A future that holds some promise for those who believe in personalized instruction 
where each student builds or constructs their learning to suit their particular style and needs. What a wonderful time 
to be a teacher! 
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Using and Evaluating Online Course Discussions 
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Abstract 

This paper explores how faculty from a small university in the northern plains use electronic discussion to support 
their online courses in various disciplines, how they evaluate student participation, and how they prepare their 
students to successfully use discussion to interact with the course content and with other students. Qualitative 
analysis of the results indicates that while faculty use electronic discussion in a variety of ways, the applications 
tend to fall in four focus areas: course content, collaboration and community building, development of higher 
order thinking skills, and course management. Results also indicate that faculty find their greatest satisfaction 
with electronic discussion stems from the increased levels of content interaction and the use of higher order 
thinking skills. 



Introduction 

As more and more learning institutions begin using the Internet to deliver courses, instructors work to 
develop better ways to bring students together, to help them make connection with others in the course and to 
enhance their learning. Asynchronous electronic discussion has become an increasingly popular tool for these 
purposes. Electronic course management tools such as WebCT and Blackboard provide threaded discussion groups 
that can be tailored for the specific demands of each course through the development of specific course related 
topics and the design of both large and small group discussion opportunities. In spite of the proliferation of online 
discussion tools, some instructors are hesitant to require the use of online discussions, either because they don't 
have a clear picture of the various ways that electronic discussions can benefit a course, or they are uncertain about 
how to effectively manage or evaluate the use of a tool that can potentially consume a large amount of student and 
teacher time and energy. Yet those who have begun using electronic discussions in their online courses have found 
that they provide structure for the course (Lieblein, 2000), build awareness of the ideas and contributions of other 
participants (MacKnight, 2000) and encourage student collaboration through information sharing and peer support 
(Sherry, 2000). Advocates point out that online discussion has the reflective characteristic of written communication 
(Northrup, 2001), and thus encourages reflective contributions to ongoing topics of discussion (MacKnight, 2000) 
and enhances the development of higher order thinking skills through a combination of human and content 
interactions (Rose, 2002; Wilson, 2002). 

Online discussion is frequently used similarly to face-to-face discussions in classrooms to encourage 
students to dig into the course content and develop understanding. Strategies such as open and small group 
discussions of case studies, debate teams, mock trials and jigsaws can be quite effective in online environments 
(MacKnight, 2000). Palloff and Pratt (1999) provide striking examples of how electronic discussion has been used to 
foster collaboration among students separated by distance and time. Some instructors provide structured questions 
to guide students' written responses to the group. Such structured questions often require the application of higher 
order thinking skills such as analysis and evaluation principles and their applications (Liaw & Huang, 2000). And 
sometimes online discussion becomes a tool for course management as it is used to document regularity and 
frequency of attendance by students in the course (Palloff & Pratt, 2001). 

As many ways as exist for incorporating discussion in online courses, just as many are available for 
evaluating participant contributions to those discussions. Most instructors who use discussion as a part of their 
courses do assign some level of participation points, but the elements they select in determining the assignment of 
those points vary widely. Some use rubrics or checklists, others provide their students with specific elements that 
must be addressed in order to gain credit in the discussion, and still others follow a pattern often seen in traditional 
classrooms, assigning points based on an overall impression of the quality of participation (McKinnon, 2000). Bean 
(1998) reported that face-to-face professors frequently rely on their impressions to determine class participation 
grades and may use an overall sense of student participation as a "fudge factor" when computing final grades. The 
professor's general impression of the frequency and thoughtfulness of a student's participation in ongoing class 
discussions could spell the difference between one letter grade or another in the course. Regardless of how 
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discussion points are determined, research indicates that grading discussion participation encourages steady 
contributions (Lieblein, 2000) and encourages student collaboration (Hathorn & Ingram, 2002). 

However, online discussion has one distinguishing characteristic that sets it apart from face-to-face 
discussion. Although, just as in a classroom discussion, the participants consider and respond to ideas and 
thoughts of others around a specific topic, the online discussion achieves substance as a written format. Threaded 
discussion postings provide the instructor with a durable record of the entire course discussion and end the 
necessity of relying on memory and general impressions to determine the frequency and quality of each participant's 
contributions. In classroom discussions, students who use active body language and a few well chosen words may 
lend the impression of substantive participation. The written record of online discussion removes nonverbal static 
and clearly separates the simple "I agree” from the contribution that provides factually supported agreement, critical 
reflection, or a penetrating question that deepens the discussion or moves it in a new direction. These are the 
elements that Palloff and Pratt (2001) refer to as substantive contributions. 

Online discussion affords new opportunities for reflective participation and collaboration as well as new 
levels of learner accountability. The pressure of being constantly visible while being physically separated from the 
rest of the class and the requirement for active learner involvement throughout the course may seem strange and 
threatening to students who have been comfortably successful in more passive traditional classroom environments. 
Instructors who wish to use the advantages of electronic discussions successfully must design enough structure 
into their discussion assignments to provide a safe learning opportunity for their students. Online professors should 
consider modeling the use of critical questioning and self reflection in the discussion (MacKnight, 2000). Berge and 
Muilenburg (2000) recommend that faculty develop guiding questions that evoke the use of higher order thinking 
skills. McNabb (2002) encourages designing discussion assignments that focus on specific goals, provide structure 
and boundaries, and encourage the use of probing and self-reflective questioning. She also suggests providing 
access to materials and resources that help students prepare for substantive discussion participation. 

How, then, do faculty successfully pull everything together? How do they use discussion in their online 
courses across various disciplines? What do they want discussion to accomplish in their courses? How do they 
prepare their students to use discussions effectively? And how do they evaluate the results? 

Method 

Nineteen faculty representing three colleges in a small northern plains university were interviewed for the 
study. All currently teach online courses and have between one and seven years experience developing and teaching 
online courses. Of those nineteen, only five reported not using discussions in their courses. The remaining fourteen 
faculty were interviewed using the following questions: 

• What do you want the use of electronic discussion to accomplish in your course? 

• Do you assign points for discussion? If so, what percentage of the course grade do they represent? 

• What factors do you consider when evaluating discussion? 

• Do you use a rubric or a checklist for evaluation? If not what do you use? 

• Do you provide specific instructions to students for conducting electronic class discussions? 

• How satisfied are you that online discussion is meeting your stated goals for the course? 

The faculty were also asked to provide copies of any student instructions and rubrics that they used and to 
provide examples of discussion postings that represented both excellent and below par student performance. 



College 


Use Discussion 


Do Not Use Discussion 


Arts & Sciences 


3 


1 


Business & Technology 


6 


4 


Education 


5 


0 



Table 1: Faculty using electronic discussion, by college. 



Results 

Information from the interviews on the uses of electronic discussions was analyzed, coded and clustered 
into four general areas: Course content, higher order thinking skills, collaboration and community building, and 
course management. 

Course content included such elements as understanding of principles, application of principles, accuracy, 
completeness, and supported by examples. 
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• Higher order thinking skills included evaluation of theories and principles, analysis of cases, problem 
solving, and moving the discussion to levels above statement of facts and application. 

• Collaboration included reading/listening to others, responding to other students ' ideas, sharing 
resources, politeness, and regularly engaging in ongoing discussions. 

• Course management included attendance, homework submissions, and quantity of postings. 

Goals 

Not surprising, interaction with course content was the most frequently cited goal by online professors in all 
represented disciplines. Professors wanted students to use electronic discussion to demonstrate their understanding 
of the content and to apply the principles they learned to real world situations and problems. 

This was followed closely by goals involving the development and maintenance of strong online 
communities. Many of the professors spoke of helping students make connections to others in the course in order to 
reduce the distance and isolation that seems to stimulate high attrition rates in many online courses. Business faculty 
in particular were interested in developing electronic collaborations that simulated the electronic workgroups many 
corporations now use regularly. 

Especially for those faculty teaching upper level and graduate level courses, the demand for evidence of 
higher order thinking skills such as analysis, synthesis and evaluation was important, but overall the terms most 
frequently used to describe desired discussion content included understanding, summarizing, supporting with 
details, and applying principles. All of these terms relate to thinking skills at the lower end of Bloom's taxonomy. 

Faculty within each college used electronic discussions to track student online attendance and 
participation, but those in business and technology courses make the most use of this feature. Likewise, this group 
of faculty was most likely to use the discussion element as a means for collecting homework. 



College 


Course Content 


Higher Order 
Thinking Skills 


Collaboration 


Course 

Management 


Arts & Sciences 


100% 


33% 


67% 


33% 


Business & Tech. 


50% 


33% 


83% 


67% 


Education 


80% 


60% 


80% 


40% 



Table 2: Goals for electronic discussion, by college 



Evaluation 

Most faculty felt strongly that assigned points for discussion participation was essential. One instructor 
stated, "I've tried it both ways, and I get much better participation when I assign enough points to make a 
difference.” His points equated to 10% of the final course grade. 



College 


Faculty Assigning 
Points 


Faculty Not Assigning 
Points 


Percentage Range 


Arts & Sciences 


2 


1 


8% - 20% 


Business & Tech. 


5 


1 


3% - 10% 


Education 


4 


1 


7% - 30% 



Table 3: Assigned percentage points for discussion, by college 



There was a wider disparity in how faculty evaluated student discussions. Only education faculty used 
formal rubrics for evaluating discussion. Remaining faculty used either informal checklists or a general impression of 
quality of participation. Those that reported using general impressions of quality and quantity also reported that they 
used discussion as a "fudge factor” when assigning final grades, giving the edge to students with consistent 
performance levels in both quantity and quality. 



College 


Rubric 


Informal Checklist 


General Impressions 


Arts & Sciences 


0 


67% 


33% 


Business & Tech. 


0 


67% 


33% 


Education 


80% 


20% 


0 



Table 4: Methods of evaluating discussion participation, by college 



81 







Faculty reported a variety of factors that they took into consideration when determining the quality of a 
discussion posting. No single element was on every list, but the most frequently used term was "substantive." 

When pressed to further define substantive, faculty provided examples such as completeness, accuracy, supported 
by facts and examples, reflective analysis, applied to specific circumstances, and adding something new to the 
discussion, or moving the discussion forward. This tends to coincide with the description of substantive postings 
developed by Palloff and Pratt (2001): "A substantive post responds to the question in a way that clearly supports a 
position, begins a new topic or somehow adds to the discussion by critically reflecting on what is being discussed or 
moving the discussion in a new direction" (pp. 79-80). Other elements frequently sited were demonstrated 
understanding, use of outside resources to expand discussion, and responds to and builds upon others' ideas. 

Faculty had a much easier time agreeing on the elements of a below par submission: off subject, 
unsupported "I agree” postings, rephrasing of other submissions with nothing new added, and rudeness. All faculty 
agreed that such submissions received no credit for participation, and three instructors stated that they responded to 
such postings by private correspondence with the author and provided guidance for more effective postings. 

Instructing Students 

Considerable disparity existed in the methods that faculty use to prepare their students for successfully 
using electronic discussions. Some faculty provided general instructions that included how to used the system for 
posting discussions and general rules of netiquette. Others provided specific instructions directly related to the 
requirements of an assignment. These instructions were more likely to provide examples of acceptable and 
unacceptable responses and to itemize specific elements (e.g., supporting data, examples, real-world applications) 
expected in a posting. Some faculty provided both general and specific instructions. Only two faculty reported 
modeling discussion postings within the ongoing class discussion. There were also faculty who reported providing 
no instructions, general or specific, to their students. 



College 


No Instructions 


General 

Instructions 


Specific 

Instructions 


Modeling 


Arts & Sciences 


0 


100% 


67% 


0 


Business & Tech 


50% 


0 


50% 


0 


Education 


20% 


60% 


80% 


40% 



Table 5: Faculty providing student instruction for electronic discussion participation, by college 



Satisfaction 

Faculty were asked to report their level of satisfaction (high, medium or low) in relation to how well 
electronic discussion were meeting their established course goals. When reporting levels of satisfaction, faculty 
comments tended to link their level of satisfaction with electronic discussion with the quality content interaction by 
their students and to the degree in which students used higher level thinking skill in their class discussions. Several 
faculty qualified their answers with statements such as, "Perhaps I set my expectations too high." And one faculty 
member reported a high level of satisfaction with online discussion and then qualified that by stating, "But I've 
changed my expectations from last year.” However, none of the faculty reported any interest and foregoing 
discussion in any future classes. The highest levels of satisfaction were reported by the instructors who 
incorporated models of desired behaviors in their ongoing class discussions. 



College 


Low 


Medium 


High 


Arts & Sciences 


33% 


33% 


33% 


Business & Tech. 


33% 


33% 


33% 


Education 


0 


20% 


80% 



Table 6: Levels of faculty satisfaction with electronic course discussions, by college 



Conclusion 

Electronic discussion is a common element in the online courses taught by the participants of this study, 
and for the most part it seems to be effective in providing a means for students to interact with the instructor, with 
other participants and with the course content. By virtue of the written format of electronic discussion, students 
interaction with the content may be more reflective and may invoke greater use of higher order thinking skills. This 
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proposition is supported by higher levels of satisfaction reported by faculty who also named course content 
interaction and development of higher order thinking skills as goals for using discussion. The written format of online 
discussion certainly provides faculty with more concrete evidence for evaluating both the quantity and quality of 
discussion participation, and this in turn, may be encouraging faculty to use more specific criteria for evaluating 
student participation, moving away from impressionistic evaluation techniques. Regardless of the factors used to 
evaluate discussion participation, the results of this study adds further support to research showing the 
effectiveness in using assigned point values for discussion participation in order to encourage regular student 
participation and collaboration. Electronic discussion also seems to be an ideal way of providing students with 
realistic models of electronic working groups currently used in businesses around the world. The small population of 
this study limits conclusions regarding the relationship between faculty satisfaction with discussion and the level of 
instruction on effective participation that they provide to students. This could be an area where faculty mentoring 
could increase the satisfaction of all. 
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Electronic Technologies in High School Mathematics Classrooms 

Aytac Simsek 

Syracuse University 
Abstract 

This paper presents the results of a qualitative study investigating the use of electronic technologies, such 
as computers and graphic calculators, in 10th grade geometry courses in a small, rural high school in 
New York. This study examines how math teachers understand the use of electronic technologies in the 
classroom to support teaching and to improve students’ conceptual understanding of mathematics. A 
discussion for the implementation of effective and supportive teaching by using technology in the 
classroom is provided. 

Introduction 

Electronic technologies, such as computer and graphic calculators, can provide teachers and students in 
High School Mathematics with the tools to develop a better understanding of mathematics. Electronic educational 
technologies are essential tools for teaching, learning, and doing mathematics (NCTM, 2000). Learning with 
technology enables students to build more meaningful personal interpretations, representations of ideas and 
simulating meaningful real-world problems (Jonassen, 1995). Research indicates that these electronic technologies are 
especially useful in developing the higher-order skills of critical thinking, analysis, and scientific inquiry (Roschelle, 
et al., 2000). Using technology tools helps students focus on decision-making, reflection, reasoning, and problem 
solving (NCTM, 2000). 

Learning mathematics with understanding is essential to enable students to use what they know, and to 
improve their conceptual understanding (Bransford, et al., 1999). Technology tools help students develop pictures of 
abstract concepts in their mind. Students’ ownership of abstract mathematical ideas can be fostered through 
technology because technology use prompts the transfer and applications of mathematical ideas into concrete 
experiences (NCTM, 2000). However, research on the effects of using technology has generally emphasized that 
“technology is not a panacea’’ (Kramarski, 2001, p.77). Jonassen (2000) argues that “students do not learn from 
technology, but technologies can support meaning making by students’’ (p. 8). Then, Jonassen (2000) emphasizes 
that students learn with technologies when technology supports knowledge construction, explorations, learning by 
doing, learning by conversing, and learning by reflecting. Clark, Hosticka, and Huddlestun (1999) emphasize that 
technology tools are powerful in the right hands, but tools can cause great harm when misused such as teaching 
lower-order thinking skills involving drill and practice. Therefore, fostering learning with technology depends on how 
technology is used and how technology is integrated into classrooms. 

An investigation of how electronic technologies should be integrated into high school mathematics 
classrooms may provide the essential information for decisions about the required conditions for successful 
implementation of technology. A qualitative research study in high school mathematics classrooms may inform how 
mathematics teachers should use technology to support teaching and to improve students’ understandings. 
Therefore, this research study examined how math teachers understand the use of electronic technologies in the 
classroom to support teaching and to improve students' conceptual understanding of mathematics. A discussion for 
the implementation of effective and supportive teaching by using technology in the classroom is provided. 

Research Purpose 

This research study considered the role of electronic technologies to support teaching and to improve 
students’ conceptual understanding of mathematics. The goal of this study was to document the distinguishing 
features of electronic technologies, the process of their integration into the current curriculum and its impact on 
mathematic education in a small, rural high school in New York. The purpose of this research was to discuss the 
results of this qualitative study investigating the use of electronic technologies in 10th grade geometry courses in 
which students and teachers have been using computer and graphic calculators. 

Method 
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Data were collected through participant observations in four different classroom and interviews with four 
mathematics teachers. Moreover, this study is a part of ongoing investigation on how electronic technologies such 
as computers graphic calculators, and projectors are integrated into their math teaching practices. This ongoing 
study consists of participant observations, document analysis, and interviews with participant math teachers. 

Research Questions 

Based on information collected from initial meetings with teachers and classroom observations it was 
assessed that electronic technologies, e.g., computers and graphic calculators had been integrated into mathematics 
classroom. 

Thus, the focusing research question was: how do math teachers use electronic technologies to support 
their teaching and to improve students’ conceptual understanding of mathematics in high schools? 

The data were collected to assess: 

• the kinds of materials and equipment being used, 

• teaching and learning perspectives on technology use, 

• mathematical concepts being presented, 

• teachers’ skills to use the technologies effectively, 

• implications of the current curriculum, and 

• math teachers’ perspectives on using electronic technologies to enhance teaching and learning. 

Analyze 

Field notes from classroom observations and data from document sources were coded and analyzed for 
categories. Interviews were transcribed, coded and analyzed. Development of the coding system was evolved 
through rereading the transcripts, and field notes. 

All of the classrooms have enough graphic calculators, and enough textbooks for all students; and these 
textbooks support use of the graphing calculators. All of the classrooms had one projector, one computer and one 
Internet connection. The school has two computer labs with access to Internet connections. 

Results from Observations 

During the lessons, teachers used transparencies, PowerPoint presentations, and graphic calculators 
instead of drawing all things on the blackboard. Students brainstormed about the concepts or questions by reading 
and discussing instead of only writing all things. When using overhead projector teacher could see students’ face 
easily, and asked questions by pointed on slides. This helps to keep students interest and improve communication. 
Teachers could save time during the lesson since they did not spent time by writing all problem statements and 
drawing complex shapes to the blackboard, and then waiting students to write down their notebooks. While teachers 
showed presentations and explained steps for solutions. Teachers guided to students their active learning process 
by solving questions together with students when using graphic calculators, by engaging students into lesson with 
question-answer method, group activities and class discussions. 

One experienced teacher used Geometry Sketchpad with the computer on teacher table and reflected the 
computer’s monitor through projector. The teacher played with Geometry Sketchpad to draw parallel lines and explain 
the different kind of lines. Since there was only one computer in the classroom students did not have a chance to 
study with computers during the lesson. Teachers believe students can explore concepts, have a better 
understanding of real life problems if they learn by doing instead of listening teacher. Students can see three- 
dimensional shapes by using graphic calculators and Geometry Sketchpad. If teachers and students use computer 
labs students have a chance of playing with Geometry Sketchpad. Thus, technology can add another dimension to 
the learning environment by doing more things then doing by pencil, and paper. 

Teachers and students used graphic calculators and a computer during lesson to drill and practice, solve 
problems, draw graphs, and discuss on the solutions. 

Results from Interviews 

Technology definitions of teachers included calculators such as the Texas Instrument TI-82, computers, 
overheads, and software e.g. Geometry Sketchpad. Teachers could use the computer as a teaching device with the 
PowerPoint presentations, the web-based reading, and a grading book. 
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Teachers stated that using technology during 82-minute periods lesson helped to keep students’ interest. 
Some problems that teachers face were that time consuming for 

• preparing of technology before class, 

• practicing new software and web sites, 

• attending inservice technology workshops, 

• learning teaching tips, 

• completing the determined topics for each semester, and 

• using computer labs during the lesson 

Teachers stated a lack of preservice training (especially for older teachers) and a lack of inservice training 
caused difficulties in using technology in the classroom. Teachers with fewer years of experience felt more 
comfortable than novice teachers during adaptation of using technology. 

Some teachers had a fear of facing a problem with computer server, programs, or overheads in the 
classroom. They wanted to have technical support for immediate feedback and training to improve their skills to use 
technological tools easily and effectively. They needed time for preparing of technology before class, for practicing 
new software and web sites, and for learning teaching tips when using technology in the classroom. 

The other important problem teachers’ state is the decisions about which technology, for what, and when 
should be integrated into curriculum. For example, they think that students are being introduced the calculators a 
little too early, and students come in high school with such weak basic arithmetic operations’ skills. 

Discussions 

Teachers in this study believed using electronic technologies allowed students to improve their 
understanding of the concepts and ideas behind mathematical thinking. However, the results of this study 
emphasized the need for teacher training on how to use these technologies effectively and realistic teaching tips to 
integrate technology into teaching practices. 

Teachers had concerns about time given to complete the determined topics for each semester, so none of 
the teachers used computer labs during the lesson. The math curriculum should be revised to let the teachers to have 
enough time to integrate technology into their teaching. Teachers should 

• develop technology materials or curriculum; 

• practice with new resources and change their teaching methods; 

• assess on what does work, and what does not work; 

• be analytical for decisions on what technology can be used for instruction and learning and how to use the 

technology; 

• match technology integration strategies with course needs; 

• have technical support for immediate feedback; and 

• improve their skills to use technology. 

Jonassen (1995) states learning in classroom should emphasize seven qualities of meaningful learning: 
active, constructive, collaborative, intentional, conversational, conceptualized, and reflective. Integrating computer 
technologies should include these seven qualities of learning to promote higher-order thinking, creative thinking and 
thus meaningful learning. Educators, teachers, and teacher education programs should try to understand these 
seven qualities and apply them during their efforts for technology integration into their teaching practices. 

Roblyer and Edwards (2000) states that elements of a rationale using technology in the classrooms are 
motivation, unique instructional capabilities, support for new instructional approaches, increased teacher 
productivity, and required skills for an information age. There are many benefits in using computers in our math 
classes (Sfard, & Leron, 1996). But, Wentworht (2002) states that mathematics teachers should rethink on their 
teaching practices when integrating technology into their classroom. Bitner and Bitner (2002) developed eight areas 
of consideration that is important to allow teachers to successfully integrate technology into the curriculum: fear of 
change, training in basics, personal use, teaching models, learning based, climate, motivation, and support. Skillful 
and motivated teachers can take advantages of technology in the learning process of their students. 

Using technology in the classroom effectively requires that teachers make good decision about which 
technology will best meet learning objectives and why. Therefore, professional development programs should 
prepare inservice teachers to use technology in the classroom effectively. 

The emphasis on integrating technology into the curriculum requires that K-10 mathematics teachers 
become experienced of instructional technology in the management of the classroom, in developing teaching/learning 
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strategies, and for the delivery of classroom instruction. Teachers must be comfortable with their skill to use 
computer software, hardware, Internet, and graphic calculators and to develop new learning strategies according to 
new technologies. 

Recommendations for Future Research 

This study was a first step in investigating on how electronic technologies such as computers and graphic 
calculators, and projectors are integrated into their math teaching practices. Recommendation for ongoing and future 
researches include: 

• conduct the study in different high schools near New York to compare and contrast the results; 

• conduct the study in different grades in mathematics classrooms to distinguish the results of different 
grades in terms of technology integration; and 

• conduct the study in science classroom to compare and contract the influence of the technology integration 
into two different subjects. 
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Designing a learning objects -based learning system: Lessons learned 

Mary Kay Alegre 

Kevin Clark 

George Mason University 

This paper focuses on lessons learned when the action learning approach was applied to the design and 
development of an online learning system using reusable content objects for a large-scale government agency. The 
theoretical model of instructional systems design (ISD) and action learning theory were combined to give students a 
framework for facing the day-to-day project management realities of: 

• effectively working as a team to solve the instructional design challenge; 

• managing client needs and expectations; and 

• designing an online learning system that utilizes reusable content objects. 

Despite the project’s complexity, the team completed a prototype of three courses and the structure of a 
course learning system. The project team was comprised of graduate students in an instructional technology 
program and they instructor. The instructor acted as the project manager of the project while the students completed 
much of the instructional design work. The students were all a part of a full-time instructional technology masters 
program. The program has students working on client-supported design projects for two semesters. 

Action Learning and The ISD model 

The action learning theory was particularly appropriate for this project because we had the components 
needed to implement the project: a committed group of team members, and an authentic task. Action learning aims to 
confront the increasing demands of environmental change and Job complexity in the workplace through promoting a 
team-based learning process (Bannan-Ritland, 2001). Teams solve the problem, and the process works to generate 
new perspectives and capitalizes on team resources. And, similar to the ADDIE model, action learning utilizes a 
systematic process to address various types of problems at the macro and micro levels. 

Documented in this paper are the processes and results of applying action learning theory to the 
instructional systems design process: 

1. Analysis - performance analysis, needs assessment, and role modeling; 

2. Design - development of essential use cases, content and task modeling, wireframing; 

3. Development - prototyping. 

Implementation and Evaluation were beyond the scope of this project, but it is worth noting that each phase 
of the process involved some form of formative evaluation, and revision. 

The Client 

The client for this project, a large government agency, wanted to improve the delivery of its current paper- 
based distance-learning courses by converting them to an online format. The client chose this course of action partly 
because of a mandate and partly because of a desire to decrease the amount of travel time and resources previously 
required to train employees. 

How Reusable Content Objects Fit Into the Project Design 

The client envisioned the use of a reusable learning objects system for this project. Learning objects, later 
termed, reusable content objects are described in more detail in the Design Phase. The concept of reusable content 
objects refers to the notion that objects such as text, graphics, video, audio, etc. can be reused in different areas of 
the learning system without reproducing the same object over and over (Wiley, 2000). Much has been written about 
the conceptual implementation of reusable content objects, but not much emphasis has been placed on designing 
and developing these systems within the instructional systems design framework. 

The project team had to consider existing specifications and standards for learning objects including 
durability, interoperability, accessibility, reusability, affordability, and manageability in making a decision on whether 
to use this technology for this particular project. The team also explored the benefits of using content objects, 
because of their potential for reusability, generativity, adaptability, and scalability. 
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Project Phases 



Analysis 

Step One: The Performance Analysis and Day-to-Day Project Realities 

The team began by conducting a brief performance analysis (PA), which is the “partnering with clients and 
customers to help them define and achieve their goals. It involves reaching out for several perspectives on a problem 
or opportunity, determining any and all drivers toward or barriers to successful performance, and proposing a 
solution system based on what is learned, not on what is typically done” (Rossett, 1999). 

The team sought out perspectives from the client. Subject Matter Experts (SMEs), and staff members. The 
team also analyzed non-human sources of data including reports, the current course content, and the current format 
and system of instruction. The team used these resources to develop a project vision describing the problem and 
opportunities, and to generate a list of drivers and barriers that would assist or hinder the team in achieving the 
vision. The project vision and underlying challenge was to “develop an online learning system for users which 
provided access to current, critical knowledge through an enhanced instructional environment.” Drivers were 
resources both technical (hardware and software) and human (SME’s, staff, and the team’s instructors, and 
advisors). Barriers identified were possible insufficient access to people: (learners, SMEs, and others) and to 
resources and information needed for a thorough needs assessment. The resulting document was presented to the 
client for discussion and approval before beginning the next step in the process. 

During the first month, while the team completed the Performance Analysis, the eventual scope of the 
project began to unfold. A series of meetings with the client revealed the possible impact of agency politics on the 
project scope. The client felt that it was important to include agencies with similar responsibilities and concerns in the 
effort to develop the online learning system. In light of this, our client began securing the interest of another agency 
in the project and subsequently began to consider inclusion of their content in the courses that were going to be 
converted. Eor the team, this raised questions about the scope of the project, the timeline for deliverables, and who 
the client was and would become. By the second month, our client was still unsure of the content and the courses to 
by prototyped. Because of this, the team scaled back the deliverables slated for completion by the end of the first 
semester. We would not fully enter the design phase; rather focus on doing a thorough needs assessment and 
analyzing system functionality. 

Step Two: The Needs Assessment and Day-to-Day Project Realities 

Upon completion of the performance analysis, the team began a thorough needs analysis process. This 
included surveying learners, SMEs, client staff members and course instructors, reviewing course materials and 
student course evaluations, as well as other sources of data. The team developed a strategy for gathering data, and 
developed instruments that would be most effective for gathering data to illuminate possible relationships between 
the performance problem and those most affected by the solution - the learner, and secondly, other system users 
(Rothwell & Kazanas, 1992). 

Because of strong indications that a second agency and its learners would be included in this project, the 
team included them in its needs analysis. The findings of the needs analysis yielded the following design 
considerations: 

1. An enhanced learning environment - including the addition of interactivity within the course and 
among learners and instructor; more appealing content and shorter courses; and the addition of 
case studies at various levels of complexity and interactivity; 

2. Current information - which describes the need to design the courses in such a way that they are 
easy to update with new information and develop the courses in a modular format; and 

3. Immediate access to information - describing a system designed for portability and printability, 
accessible anytime from anywhere. 

In meetings with the client during the needs analysis, the team discovered that the client was in the process 
of developing a web site and structure for handling course management functions, including student login, 
registration functions, and course management capabilities. All of it was to be backed by a database. With this 
finding, the scope of the project changed. The team no longer needed to develop an entire learning system from login 
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to course completion, rather only needed to focus on course functionality and converting three courses to an online 
format. 

This finding also had implications for reusable content objects. Because the client was building a course 
management system with a database infrastructure, the feasibility of designing the courses using a reusable content 
objects database significantly increased because the technology was already in place. The team more thoroughly 
researched reusable content objects during this phase, and along with the client, made decisions related to their 
design. 

Design 

Step Three: Development of Personas 

While each team member finalized his/her section of the Needs Assessment Document, sub-teams 
developed role models or personas for those who would be using the system. The method of creating personas is 
simple but extremely powerful - “make up pretend users and design for them” (Cooper, 1999). Personas have goals, 
characteristics and personalities and can even have a name. These characteristics are based on research and 
personify a particular class of user. Once personas are created, design decisions are based less on design team 
hunches or opinions, or what “all users’ might want and need, and are based more on a particular user that represents 
a single category of user. The team built one persona for each of the different categories of users: learners, 
instructors, course administrators, and, learner supervisors. 

Step Four: Development of Essential Use Cases 

For the four personas, the team developed essential use cases. Use cases provide a way to build system 
functionality based on a usage-centered design approach that responds to user needs. It is a model that focuses 
attention on user intentions and simplified system responses (Constantine & Lockwood, 2001). To create the 
essential use cases, the team developed well-defined user tasks and matched them with system responses. These 
tasks were pulled from the Needs Assessment Document. Each case was described in abstract and in technology-free 
terms to allow for alternative design solutions during the next step of the process when team members determined 
system functionality and built wireframes. 

Step Five: Determining System Functionality and Building Wireframes 

The team, along with its programmer, worked very closely with the client to build a system compatible with 
the system being built to administer and house the courses. With results from the Needs Assessment, as well as 
knowledge of user intentions and goals, the team was able to identify those system functions necessary to satisfy 
user needs. Some of these functions were already available on the client’s system, the others would need to be 
added. Once these functions were determined, they became the basis for course navigation. With functions 
identified, and content developed and ready for placement in the course, the team began building wireframes. 
Wireframes are visual representations of each of the system functions. When used together with a navigation map, 
they show both how the interface is laid out, in its most simple and general terms, and how the system will be 
navigated based on user interactions (Constantine & Lockwood, 2001). The wireframes and course navigation maps 
along with the course content were provided to the team programmer who built a database structure using web-based 
applications such as ASP and Cold Fusion. 

Step Six: Reusable Content Objects and System Design 

Before chunking the content, and while the team decided on system functionality, and system navigation, 
and built course wireframes, we continued to research learning object systems, and the feasibility of using learning 
objects, or reusable content objects as the foundation of our course structure. Discussions continued with the client 
about if and how to approach the creation of reusable content objects. We needed to consider conforming to official 
standards called the Sharable Content Object Model or SCORM. SCORM is a “set of specifications for developing, 
packaging and delivering high-quality education and training materials whenever and wherever they are needed” 
(SCORM, 2001). SCORM comes from a U.S. Government's initiative in Advanced Distributed Learning (ADL). 

The team learned that a full-fledged learning objects system that complies with these standards would 
probably be beyond the scope and budget of this project. The client proposed a different approach so that content 
could be reused among and between courses. The decision was made to develop course and system specific tags, 
based on the tags indicated in the standards. This solution would be cost-effective and allow for future growth and 
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compliance if desired. This level of tagging, however, could not be completed in the short time that remained in the 
second semester, so alternate plans were made to simply chunk content according to topic, using naming 
conventions to identify the content’s placement in the course. Each chunk was added to the database and would 
serve as the foundation for a reusable content object system. 

Development 

Step Seven: Prototyping 

During the development phase, the system, its functions, and content got a “look”. The user interface was 
created with the help of a graphic designer. The team’s programmer added the graphics to the database, and built the 
infrastructure for and populated an active, high-fidelity prototype (Constantine & Lockwood, 2001). An active, high- 
fidelity prototype is one that functions to the extent necessary to show how the real system will work, and one that 
closely resembles the user interface. 

Conclusion 

The prototype, along with a description of all phases of the instructional design process, was presented to 
the client upon completion of the second semester. The team summarized its experience of working as a team and 
putting action learning theory into practice to solve the ISD problem. The team successfully fulfilled its vision of 
building three course prototypes that were interactive, appealing, portable and printable, and easy to update. The 
courses, now in a web format, will provide students of land and realty management an enhanced learning 
environment. This environment was created only after conducting a thorough needs assessment, and through the 
team’s understanding of and commitment to usage-centered design, and planning in great detail the course 
functionality based on user needs. The prototypes will need to be tested, and will go through several iterations 
before they become the new method of training for emerging managers of the two government agencies. 

Lessons Learned 

As a result of working on this project there were a few lessons that we learned: 

• Establish the database structure with the assistance of a database engineer prior to beginning 
design on the essential aspects of the system. Because the database system and the online 
learning system were viewed as two different systems, issues of compatibility were overlooked. 

This proved to be a tremendous challenge when the time came to integrate the two “systems” into 
one large-scale online learning system with a database infrastructure. 

• In addition to determining the database structure, you should also fully investigate the details 
involved in implementing the technology. In this case, the client wanted to implement a cutting 
edge technology such as learning objects. Because of the lack of time and resources, neither the 
client or designers were able do dedicate the time required to fully investigate all of the implications 
involved in designing for a learning objects -based system. 

• Have client and team work closely together on the design and development; communication is key. 
The client and the design team should work very close together so that each is aware of the others 
needs, vision, and resources. This will eliminate the possible of duplication of efforts as well as the 
need for drastic and sudden changes in project goals. 

• Create prototypes to test out ideas and approaches. In order to help the client and designers gain a 
clearer understanding of what is to be achieved, prototypes enable both parties to put their ideas in 
context; giving abstract ideas more details. 

Many of the lessons learned boiled down to basic common sense and sticking to the Instructional Design 
Process. The caveat is that when you are attempting to introduce or utilize new technologies is especially imperative 
that the processes are followed and that no steps are skipped or ended prematurely. Working with a new technology 
does not afford you the opportunity to make changes on the fly unless the designers and the client are extremely 
aware of the technology’s capabilities and constraints. 
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Design of Faculty Development Workshops: Attempting to Practice 

What We Preach 
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Abstract 

The purpose of the presentation is to provide recommendations for designing and implementing faculty 
development programs on technology integration. The recommendations are based on instructional strategies as 
related to faculty development in higher education. Research data will show that the strategies were comprised of 
two components or factors. Lastly, the results demonstrated which instructional factor that had the most effect on 
teaching impact and workshop satisfaction. 



Introduction 

Today’s university faculty face many problems from decreased funding and resources to increased diversity 
in student populations that make it difficult to maintain the traditional faculty roles (Davidson-Shivers, 2002; 
Morrison, 1996, Miller, 1995). The traditional workload of higher education faculty at four-year institutions has been 
based on teaching, research, and service. There is a tendency within universities using the Carnegie system that the 
higher the level of university, the stronger the emphasis is on research (Davidson-Shivers). Even with two-year 
colleges, faculty are encouraged to conduct research in addition to their primary focus on teaching (Tsunoda, 1992) 

According to Rutherford and Grana (1995), faculty roles in teaching are shifting from knowledge gatekeepers 
to knowledge facilitators. This change in roles is based on a trend toward active and less authority-dependent 
learners and students’ access to information through technology. In addition, faculty are undergoing change due to 
the availability of technology for classroom use (Johnstone & Krath, 1996; Young, 1997). Even though this change is 
eminent and some faculty have made the transition, there are still many who have not (Armstrong, 1996). 

This resistance to change may be due to a variety of factors: 1) faculty may lack the skills to use technology 
appropriately; 2) lack of institutional support (Horgan, 1998) and new roles of the professor are not rewarded 
(Armstrong). New roles currently being suggested are those of course developer and facilitator, rather than 
instructor. According to Horgan, incentives such as promotion and tenure are still based on traditional roles. Einally, 
faculty development efforts may not be well received because they are not well designed. 

The traditional faculty development, such as workshops, of the 1970s and 1980s were viewed least effective 
(Maxwell & Kazlauskas, 1992) and that they may have been misdirected (Blackburn, Boberg, O’Connell, & Pellino, 
1980). In their review of literature, Davidson and Rasmussen (1994) found few ID models specifically developed for 
faculty development. Although not all models are appropriate for every situation or setting, some underlying ID 
principles do transcend disciplinary boundaries. However, these ID principles serve as guides in our faculty 
development efforts. 

Analysis Phase. 

Our faculty development efforts began with the analysis phase. Because of a short timeframe for planning, 
the goal of the workshops was based on the desire of the university’s administration to upgrade faculty’s use of 
technology tools in teaching. Being faculty members at that same institution, we knew the environmental settings 
and the faculty. We had also served on the University's Committee on Teaching , developed faculty seminars in the 
past, and created and coordinated the Educational Technology Poster session involving faculty from across the 
university. Based on our prior knowledge and experience, we began developing the workshops on PowerPoint^M. We 
set the maximum of 20 participants per workshop. 

Concurrent Design and Development Phase. 
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During the concurrent design and development phase, we planned and developed the instruction in an 
integrated manner. We met several times to plan our workshop as follows: 

1. identify the sequence from simple features to more complex ones. Provide simple definitions, overview 
of procedures, and establish the tone of the workshop. 

2. create the advertisements and establish a faculty registration system that encouraged faculty to bring 
their own teaching materials. The logistics of breaks and lunch were also delineated. 

3. prepare the instructional strategies, emphasizing demonstration and hands-on application. Included 
would be the development of several PowerPointTM presentations and handout materials. 

4. Select and purchase additional support materials of books and videos. 

5. Divide the instructor tasks for developing our PowerPoint™ presentations, handouts, etc. and 
conducting the demonstrations. 

6. determine the location, obtain administration approval and purchase PowerPoint^M for the computer lab 
to meet licensing requirements. 

Evaluation Planning. 

Evaluation Planning began simultaneously with concurrent design. We planned for a simple survey on the 
effectiveness of the instruction. We reviewed and revised our materials by conducting trial runs prior to the initial 
workshop. 

Implementation Phase. 

Eor Implementation phase, we considered the roles of facilitator as well as the participants. The workshop 
allowed for a high-gain of information by providing job aids, hands-on exercises, and personalized feedback. In 
addition, we created a relaxed atmosphere by establishing dialogue between instructor and participants and among 
faculty participants early on and using humor appropriately during the workshop. The type of materials used and the 
style of the various PowerPoint slide presentations added variety. Einally, we added variety by changing which 
instructor would demonstrate and which would provide assistance and feedback throughout the day. 

This research is a result of surveys completed and analyzed after the conclusion of the workshops. The 
objective of the analyses was to investigate which instructional strategies that influenced the participants’ overall 
workshop satisfaction as well as their professional development. 

Methods 

As previously discussed, three components of the design and development phase included: identifying the 
appropriate sequencing for the workshop, planning instructional strategies, hands-on applications, providing 
support materials, and delineating instructor responsibilities for presenting content and software demonstration. The 
survey instrument was created in order to evaluate the effectiveness of the instructional strategies, the impact of the 
workshop on the participant’s teaching development and skills, and overall satisfaction with the workshop. 

Participants 

All participants were asked to complete the survey at the end of each workshop. Within three years, six 
introductory and four advanced workshops were offered to university faculty. Eighty-nine faculty members 
participated in the introductory workshops, while 46 participated in the advanced. One hundred thirty-five faculty 
members participated in the workshops over the three-year period. The introductory workshops contained a range in 
participants from 1 1 to 19. The advanced had a range from seven to 15. All workshops were sponsored by the 
university and conducted in the campus computer lab. 

Measurement Items 

The 20-question survey instrument was specifically designed to evaluate feedback on: technical 
frustrations, handouts, video presentation, textbook, order of topics, hands-on activities, instructor’s estimate of 
participant knowledge, pace of workshop, constructive feedback, and organization. Ten items were developed to 
investigate the participant’s reaction to the workshop’s instructional strategies. Two questions were developed to 
seek feedback on the workshop’s effect on how the participant will function professionally and its impact on 
teaching. Lastly, two questions were developed to assess participant satisfaction and willingness to recommend the 
workshop. 
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Fourteen of the 20 questions utilized a five-point Likert scale. Two of the fourteen questions did not have 
weighted values assigned to the Likert response scale (instructor’s estimate or participant knowledge and pace of 
workshop). Those items were not utilized for the analyses. Six open-ended questions were added to collect specific 
feedback on the structure of the workshop and instructor style. The open-ended items were also not included in the 
analyses. Participants were asked to circle the letter response (i.e. a, b, c, d, or e) that best reflected their opinions to 
each statement. For example, question one stated: 

In general, this workshop: 

(a) (bj (cj (dj (e) 

could have a major had little or no effect 

effect on how I will function 

professionally 

The (a) response was considered favorable, (c) was considered neutral, and the (e) response was 
unfavorable. All responses were recoded numerically and assigned weighted values appropriate for discrete 
statistical analyses: a=l,b = 2, c = 3, d = 4, and e = 5. 

Statistical Design 

Statistical Program for Social Sciences 9.0 (SPSS Inc., 1999) was used to analyze the data. Factor analysis 
was conducted to investigate validity of the instrument and Cronbach’s alpha was used to test for reliability. 

Multiple regression was utilized to investigate the influence of the instructional strategies on two dependent 
variables: teaching impact and workshop satisfaction. 

Results 

Due to two items not having weighted values assigned to the Likert response scale, only eight items 
investigating the reaction to the instructional strategies were analyzed. The items were subjected to factor analysis 
and rotated using a Verimax rotation. Factor analysis was used to reduce the number of variables required to describe 
or explain the items comprising the instructional strategies construct. Additionally, it was conducted to assess the 
construct’s psychometric properties (Rosenthal & Rosnow, 1991). Consequently, two factors were rotated using the 
Verimax procedure. Table 1 shows the instructional strategies investigated, the two interpretable factors, and the 
factor loadings. Factor one accounted for 40.4% of the item variance, and Factor two accounted for 6.7% of the item 
variance. The item assessing the response to the hands-on activities did not load on either factor. 

Table 1: 

Factor Analysis for Instructional Strategies 



Item 


Factor 1 Factor 2 


Factor 1 


Workshop organization 


.90 


Flandouts were helpful 


.83 


Order of topics 


.74 


Benefiting from technical frustrations 


.56 


Textbook was helpful 


.51 


Workshop was designed for constructive feedback 


.45 


Factor 2 


Flelpfulness of video 


.62 



The factor analysis clearly indicated two salient constructs when investigating the participants’ response to 
the instructional strategies. Factor one appears to relate to some components of the concurrent design and 
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development phase as well as the implementation phase: identifying the appropriate sequencing for the workshop, 
planning instructional strategies and support materials, and delineating instructor responsibilities for presenting 
content and software demonstration. The researchers referred to factor one as Instructional Strategies. While factor 
two does relate to the benefit of support materials, it is a type of support material in a non-print format that was 
different from the other materials provided during the workshops. The researchers referred to this factor as Video 
Support. 

Cronbach’s alpha was used to test the internal-consistency reliability for factor one. The closer the 
reliability coefficient (Cronbach’s alpha) is to -i-l.OO, the more reliable the dimension. The closer the reliability 
coefficient is to 0, the less reliable the dimension, (Crowl, 1996). Reliability coefficients of approximately .85 or higher 
may be considered dependable psychological tests, whereas in experiment research, instruments with much lower 
reliability coefficients may be accepted as satisfactory (Rosenthal & Rosnow). According to Ary et al. (1996), if 
measurement results are to be used for making a decision about a group or experimental research purposes, a lower 
reliability coefficient (in the range of .50 and .60) might be acceptable. The results of the reliability analysis yielded an 
alpha coefficient of .83 for factor one. Therefore, the first factor appears to be valid and reliable. Factor two was 
comprised of only one item with a considerably high loading. However, due to the one item loading it was not 
subjected to reliability testing. 

The factor analysis on the items assessing participant satisfaction and willingness to recommend the 
workshop also resulted in one factor being extracted. The items accounted for 64.2% of the variance and the factor 
loadings were .82. The reliability results indicated an alpha coefficient of .78 for the factor lending support for validity 
and reliability. This factor was referred to as Workshop Satisfaction. As previously mentioned the responses were 
recoded numerically and assigned weighted values appropriate for discrete statistical analyses: a = l,b = 2, c = 3, d = 
4, and e = 5. Therefore, after recoding the lettered responses 1 = favorable, 3 = neutral, and 5 = unfavorable. Table 2 
indicates the scale distribution for the four factors. 

The items that were developed to investigate for the workshop’s effect on how the participant will function 
professionally and contribution to skill development were also subjected to factor analysis. One factor was extracted 
and both items accounted for 52.5% of the variance and the factor loadings were .72. The results of the reliability 
analysis yielded an alpha coefficient of .68 for the factor. Therefore, the factor also appears to be valid and reliable. 
The researchers referred to this factor as Impact on Teaching. 

Table 2: 

Factor Scale Distribution 



Construct 


N 


Mean 




Instructional Strategies 


74 


1.48 


.50 


Video Support 


84 


1.67 


.76 


Workshop Satisfaction 


82 


1.27 


.49 


Impact on Teaching 


84 


1.66 


.59 



Two constructs were extrapolated from factor analyzing the instructional strategies: Instructional Strategies 
and Video Support. Two regression analyses were then used to explore which instructional strategies influenced the 
dependent variables: (1) Workshop Satisfaction and (2) Impact on Teaching. The Pearson correlation in the first 
regression indicated that Instructional Strategies correlated with Workshop Satisfaction (r =. 790, n = 71, p<.01) as 
well as the Video Support (r = .287, n = 71, p<.01). The results of the regression (Table 3) indicate the Instructional 
Strategy factor explained almost 64% of the variance in Workshop Satisfaction. 

Table 3: 

Regression Analysis for the Workshop Satisfaction Factor 



Variable 


r2 


B 


SE 


Beta 


T 


Sig. 


Constant 




.133 


.120 




1.101 


.275 


Instructional Strategies 


.638 


.864 


.085 


.856 


10.217 


.000 


Video Support 




.009 


.059 


-.134 


-1.604 


.113 



Note: Dependent variable = Workshop satisfaction 
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Indicated in the second regression was the Pearson correlation that revealed that both the instructional 
strategies(r = .589, n = 72, £<.01) and video support (r = .384, n = 72, p<.01) correlated with Impact on Teaching. 
Specifically, the regression indicated that the factors explained 36% of the variance on the Impact on Teaching 
dependent variable, and that the Instructional Strategies factor significantly predicted Impact on Teaching. Table 4 
depicts the results of the first regression. 

Table 4: 

Regression Analysis for the Impact on Teaching Factor 



Variable 


r2 


B 


SE 


Beta 


T 


Sig. 


Constant 




.480 


.196 




2.445 


.017 


Instructional Strategies 


.360 


.651 


.136 


.526 


4.790 


.000 


Video Support 




.114 


.095 


.131 


1.196 


.236 



Note: Dependent variable = Impact on teaching 

Discussion of Results 

The results of our analysis clearly indicate that the workshops were successful. The impact of instructional 
strategies, planned and employed, produced a strong satisfaction rating for the workshops. Using our faculty 
development workshop model represents the notion that providing appropriate instructional strategies for what 
faculty need and enjoy can result in positive ratings. 

These PowerPoint workshops were organized around a relaxed atmosphere whereby instruction was 
chunked into instructor-led demonstration, hands-on activities for the participants, and discussion amongst 
participants and between participant and instructor. Most faculty workshops are instructors talking at participants, 
which is different from demonstrations, with little active involvement of participants. However, these participants 
were given the opportunity to work on their own materials throughout the workshop. What also may have been 
significant about the organization of these workshops was that faculty were encouraged to bring their own teaching 
materials, which brought relevance to the instruction. Additionally, we also realized that we there was a need to 
develop or supply concrete instructional support materials that would facilitate faculty in learning the steps involved 
in producing a slide show. Hence, we created short demonstrations of how to build PowerPoint in a step-by-step 
manner. Each demonstration was followed by opportunities for practice. Finally, sequencing of the instructional 
activities from demonstrating steps to practicing the steps as well as switching instructors for various chunks of the 
instruction provided variety. Finally, the majority of the support material was print based that allowed participants to 
reread information on steps demonstrated as they worked on their slide presentations. 

The Video Support may have been singled out as the other factor for Workshop Satisfaction due to its 
novelty effect. With the video, we chose to show only those portions of the video that explained certain aspects of 
PowerPoint, such as Pack and Go, in a very efficient manner. Limiting the amount of what was viewed at any given 
time also perhaps helped faculty maintain interest. Hence, keeping the use of this passive type of learning to a 
minimum. Likewise, using video may have added variety to what faculty traditionally are provided; most faculty 
workshops rely heavily on instructor presentations supported by print-based materials. 

Instructional strategies also explained Impact on Teaching but to a lesser degree than Workshop 
Satisfaction. It is not surprising that it indicated a less strong explanation since we were asking the participants to 
speculate on how the effect of the workshop would project onto their future teaching. However, in planning the 
workshop, we realized that in order for these participants to continue to use what they had learned about PowerPoint, 
they would need support material that could serve as reference materials for use at a later date. Our handouts as well 
as the book served that function. 



Summary 

Higher education faculty realize that there is a need for change due to pressures from outside and within 
their institutions. While some faculty may be reluctant to change, others willingly take advantage of opportunities for 
professional development when offered. However, they are only willing to do so when, and if, faculty development 
efforts are focused on improving their knowledge and skills and when these development activities are effective. Our 
initial data suggests that our ID planning and implementation guidelines provide an avenue for developing programs 
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that enhance faculty teaching and instructional skills. Further exploration on the use of our ID guidelines needs to be 

expanded to other areas of faculty development as well as to other higher education institutions. 
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CAMS: Automating Data Collection to Ensure Program Quality 
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San Diego State University 
Abstract 

When fully developed, San Diego State University’s Continuous Assessment and Monitoring System ( CAMS) will 
allow faculty in the College of Education to conduct a broad range of self assessment activities. The system is not 
meant to measure competency but it can interface with pertinent college and/or university systems that track such 
information. CAMS has the potential to be the vehicle by which the College of Education can periodically but 
continuously determine that its seven departments (among them, the School of Teacher Education, Special 
Education, and Educational Technology) are NCATE-compliant, on-course, stringent, market-responsive, in 
compliance with all state laws and regulations, and motivational. 

CAMS: An Introduction 

Accountability is the watchword for educators today. In the K-12 environment, accountability is reflected in 
the public’s clamor for improvement indicators that range from gains on student achievement tests and college 
entrance exams, to fewer instances of both petty and serious schoolyard crime, to reductions in absenteeism and 
transiency. At the college or university level, accountability indicators include graduation rates and time to 
graduation; program enrollment choices and perceptions of program quality; access to and use of support services; 
and employer “satisfaction” with the “product” - graduates. 

Accountability increases the information demands placed upon our educational system. Regardless of size 
or student demographics and no matter how organized, our schools depend on integrated information systems to 
manage their data loads and to respond to the community’s desire for immediate access to detailed information about 
school life (e.g., student performance, the costs of running extracurricular clubs and activities, a school’s readiness 
for incidents of violence). Unfortunately, empirical evidence relative to the implementation and effects of such 
systems is fairly lean. There’s little to suggest that accountability systems of any type factor into policy making tied 
to accreditation mandates, or evoke change in overall data management, program operations, organizational structure, 
or personnel practices (hiring, deployment) (Bober, 2001). 

Accountability is central to the day-to-day functioning of San Diego State University’s College of 
Education, which has a strong focus on urban education, technology, and preparation of practitioners for work with 
diverse populations. Its unique characteristics showcase the ways in which accountability issues come into play: 

• Each year, between 700 and 800 teacher candidates graduate from the COE’s several teacher preparation 
programs. That figure represents -7% of the CSV total. 

• COE students may earn a variety of master’s degrees and professional certificates - from rehabilitation 
counseling to literacy. 

• Currently, ~80 COE students are enrolled in one of two joint doctoral programs. 

• The COE employs -100 tenure or tenure -track faculty, and -100 part-time or full-time lecturers. 

• Eaculty are active grant writers; total yearly awards (grants and contracts) exceed $15 million, on 
average. 

The SDSU/COE recognizes the need to continuously capture informative data that allows for both strategic 
and tactical decision-making. The Continuous Assessment and Mentoring System (CAMS) is an innovative tool for 
doing Just that. Structurally, CAMS is a set of web-deployed (and cumulative) surveys that capture perceptual data 
from students at no less than four strategic points during their academic careers: at the time they complete 
prerequisite courses; about midway through their program of study; when they enroll in their capstone experience; 
and about one-year post-graduation. But the system is more than a mere collection of respondent opinions, 
reactions, and beliefs. At its core are several relational databases populated by a host of program-specific data, 
including objectives for each of the courses students must or may elect to take and the ways in which objectives 



99 




build across course strands.' As important is that CAMS, when fully deployed, will be able to access data available 
on other college or university-level computer systems. 

In the end, then, CAMS can contribute to the COE’s vitality and relevance. The system will generate 
student profiles that portend program quality. Sophisticated searching and sorting routines will allow faculty to 
provide advising services tailored to student interests and needs, ensure that course strands maintain their integrity, 
and modify internship/field experiences based on workplace demands. Perceptual and program-level data will be 
combined in ways that allow for a host of tactical and strategic decisions: creating new courses or revising existing 
ones; personalizing academic advising; and adjusting faculty and staff resource allocations. 

Of course, anonymity and confidentiality will be protected at all costs. Integrated into the system will be 
dynamic security features to ensure that only authorized users view sensitive data, package information, and 
generate reports. Once fully operational, CAMS will allow the COE to be responsive and accountable to its many 
constituents. 



CAMS: Verifying the Need by Exploring Institutional Processes 

There’s extensive support in the literature for a CAMS-style system. Institutions of higher education are 
under increasing pressure to demonstrate value: to students, to faculty, to administrators, and to the larger 
community. In 1998, the National Center for Public Policy and Higher Education used established benchmarking 
procedures (see; http ://w ww .g ut. edu.au/law/benchmarking/overview .html ) to produce a state-by-state report card 
(published in 2000) that compared post-secondary institutions across several dimensions - among them, preparation, 
participation, affordability, completion, benefits, and learning. Though faculty assumptions drove at least some of the 
data analysis, results appear to confirm longstanding concerns about the accuracy of institutional assessment 
practices, actual availability of data, and the ways in which systematically collected data inform funding and other 
core decisions (see; http://measuringup2000.highereducation.org/highlights2.htm ). 

Increased competition for funds tends to propel accountability - whether the effort is mandated or 
voluntary. 

• Voluntary accountability is evidenced by the Malcolm Baldrige National Quality Award (Acaro, 1995), 
which promotes self-assessment and organizational excellence. The educational strand forces academic 
leaders to take a broad-based look at program quality. Recipients are judged to have achieved 
distinction in leadership, strategic planning, customer and market focus, information and analysis, 
human resource focus, process management, and results. 

• Mandated accountability is evidenced by accreditation, a sanctioned evaluative process for measuring 
institutional (or program) strengths and weaknesses. Accreditation is a monetary gatekeeper of sorts, 
with nonaccredited institutions largely excluded from traditional funding streams. The purpose of 
accreditation is well articulated by the Commission on Higher Education - a unit of the Middle States 
Association of Colleges and Schools. 

• Accreditation is the educational community’s means of self-regulation through quality assurance 
and improvement. The accrediting process is intended to strengthen and sustain the quality and 
integrity of higher education, making it worthy of public confidence and minimizing the scope or 
exercise of external control, ( http ://msache . org/poevpre .html ) 

Mandated accountability is also reflected in state laws and regulations. Fully integrated into the newly 
revised California Master Plan for Education, for example, are regulatory procedures for ensuring that students are 
academically prepared for the workforce; are enrolled in programs that are fiscally responsible; have equitable access 
to resources they need to be successful (academic, social, logistical); and are taught by competent faculty intent on 
growing the knowledge bases of their respective disciplines. Here, CAMS can serve as a preemptive mechanism to 
ensure both compliance and value added. 



' In the Educational Technology program, for examp le, strands fall out as follows: instructional design (two courses); 
inquiry and multimedia development (four courses); research (three courses); and organizational leadership (two 
courses). 
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And CAMS directly responds to the revised standards and criteria promoted by the National Council for 
Accreditation of Teacher Education (NCATE), the accrediting body for SDSU/COE. As the heart of the “new” 
NCATE is a clear focus on performance-based outcomes. Programs must be able to demonstrate that a) their 
graduates have the skills and knowledge needed to educate all learners and b) faculty and staff participate in 
continuous self-assessment for improvement based on accurate and dependable data. The agency’s newly revised 
Unit Standard 2 specifically targets assessment systems and program evaluation (see: 
http://www.ncate.Org/standard/m stds.htm ). The idea is to demonstrate integrated processes that: 

• mirror the conceptual frameworks defined by professional and state standards. 

• collect data through diverse methods that are systematic and fair . . . and which continually cross-check 
for data validity. 

• provide comprehensive information about program quality, operations, and student performance. 

• enable the unit to conduct meaningful assessment that leads to credible (and data-based) improvement 
in whatever areas fall below acceptable levels. 

CAMS: Verifying the Need by Exploring the Researeh Base 

A comprehensive accountability system must necessarily be hath, data and information driven. Data, 
according to Ramer and Snowden (1994), are collections of facts and figures, while information suggests answers to 
questions provided by a person or agency as a service to others. In essence, then, CAMS is a data-driven system 
that provides easy access to timely information that has relevance and purpose , with the intention of empowering its 
users. It’s integrative by its very nature. 

Telem’s (1996) depiction of the K-I2 setting has broad applicability to the post-secondary 
(college/university) environment. He suggests that most schools are bureaucratically, professionally, and politically 
complex organizations, characterized by a “loosely coupled nature” (p. 87). Staff members, though interdependent, do 
not exhibit strong interpersonal ties. Moreover, the school’s subsystems and the functions within them are fairly 
insulated from one another. Leadership is diffused rather than concentrated; there is high respect for professional 
autonomy (Telem, 1999; Telem & Avidov, 1994). 

Introduction of a student-focused accountability system such as CAMS has the potential to impact a 
variety of instructional and administrative roles and responsibilities. In essence, thoughtfully -planned 
implementation may contribute to stronger educational leadership, to an environment that encourages self- 
improvement, and to School renewal. 

Positive change in roles and relationships is one of several outcomes associated with system 
implementation. When well-deployed, the data are integrative, timely, reliable, and easily -accessed . . . and can 
significantly impact interrelations among the several functionaries that comprise a college administration: staff, 
adjunct faculty/lecturers, tenure -track faculty, tenured faculty, program chairs and coordinators, and a Dean. Results 
drawn from Telem’s year-long study of an 800-student high school have implications for the SDSU/COE: 

• Issues that were only occasionally or randomly dealt with before accountability system implementation 
were regularly addressed “according to predefined mandatory regulations including mandatory 
timetables” (p. 203). 

• Vertical and hierarchical interrelation patterns among administrative functionaries “tightened up” (i.e., 
relationships became stronger, interaction was more frequent, and individuals became less 
autonomous) following system introduction. 

• Horizontal interrelations among faculty also “tightened up” - reflecting themselves in higher 
competition tempered by more intensive cooperation, teamwork, and joint planning. 

• The system, even when just partially assimilated into daily school life, created patterns of interaction 
reflective of the business environment. 

A CAMS-like accountability system is, in fact, a social enterprise. A bevy of social consequences must be 
considered during system planning and as the product is deployed. Student perceptions of it - from relevance to 
academic planning to impact on academic progress - can affect the quality of the data collected. The system is 
doomed to failure if students feel that their input favors or disadvantages them in some way - grades, selection for 
special/unique opportunities, selection for internships or assistantships, registration priority. As a result, they may 
opt not to respond, or provide information that falls into three distinct categories: socially correct, politically 
correct, or error-prone. The research is rife with descriptions of unanticipated effects that warrant caution among 
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savvy system “operators”: the sense of being over-surveyed; expectations (often, unrealistic) regarding use of 
results; concern with privacy and data security; concern about (and confusion over) anonymity and confidentiality 
(Abbott, Wulff, Nyquist, Ropp, & Hess, 1990; Anton, Dilla, & Fultz, 1997). 

But system integrity also depends on engaging/acM/ty. The MSACHE framework earlier referenced 
advocates total faculty involvement at all stages of system development: design, prototyping, testing, full 
deployment, ongoing maintenance. Faculty commitment is predicated on several assurances: 

• That the system’s purpose is improved teaching and learning - not loss of academic autonomy. 

• That system activities are not burdensome in terms of workload. 

• That comprehensive training and ongoing support are available. 

CAMS: Research Issues/Methods and Analytical Techniques 

CAMS is a comprehensive project, designed to affect the ways in which all core constituent groups 
(students, faculty, administrators, staff) think about accountability and data-driven decision-making. Clearly, the 
research/evaluation design must be robust, systematic, and responsive to both interim and impact outcomes and a 
mix of data collection and data analysis strategies. Undergirding this effort, then, is a four-level management-oriented 
approach known as CIPP (Stufflebeam, 2000), structured to help high-level program staff make good decisions when 
presented with viable alternatives (Worthen, Sanders, & Fitzpatrick, 1997). CIPP’s flexibility accommodates both 
formative (improvement-oriented) and summative (judgmental) assessment; as a noncumulative model, it allows for 
one “class,” group, or level of decisions to be made independently of others - which is likely the case here. CIPP 
attends well to the different areas of complex decision-making inherent to this project’s intent: Context (which 
informs planning decisions); /«/>«< (which informs structuring decisions); Process (which informs implementing 
decisions); and Product (which informs recycling decisions). Integrated into the CIPP framework will be tenets of the 
Concerns Based Adoption Model (CBAM), an approach that attends to participants’ responsiveness to change and 
innovation (Rogers, 1985). 

CAMS development follows a four-phase process. Phrase 1 is ongoing needs assessment, which calls for 
completion of the following tasks. 

• Validating the clusters of questions that currently characterize each of the current surveys. 

• Validating the survey items themselves (and the usefulness of the data they’ll generate). 

• Determining ways to rephrase questions that are meant to serve a cumulative function. 

• Determining ways to triangulate data, thus generating reports that truly address tactical or strategic goals. 

• Determining database and report layout and format - to ensure data accuracy isn’t compromised by 
aesthetics. 

• Determining ways to couch directions to reduce confusion on the part of survey-takers and report 
producers 

• Seeking strategies for “institutionalizing” the process into the fabric of the department and college. 

• Seeking alternative strategies for ensuring data integrity - and anonymity and confidentiality. 

• Identify viable vendors (to reduce the need for “reinventing” programming tasks/processes 

Phase 2 is largely devoted to product design and pre-rollout. Major tasks include: 

• Completing technical specifications, including how different technologies (and college-or university-wide 
systems) will seamlessly interface. 

• Developing a marketing strategy (for faculty and students). 

• Preparing strategies for acting on results (tactically or strategically). 

• Formalizing the evaluation process. 

Phase 3 is where development scad formal rollout occur: 

• Building each of the four surveys (including features and functions that allow for advanced searching and 
report generation). 

• Developing and launching product orientation activities. 

• Establishing protocols for reporting and resolving technical problems (navigation, etc.). 

• Designing and prototyping an ongoing maintenance program (that addresses both the system and the 
‘content’ at its heart). 

• Developing update procedures (changes in course objectives, changes in courses themselves). 
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• Conducting a small pilot test, and revising product elements based on results 

• Finalizing a process for ramp -up (tailoring the system to match Department-specific needs). 

Finally, Phase 4 centers on marketing and dissemination, as well as monitoring system use. 

It is premature at this time to devise a complete research strategy, but clearly, research and evaluation 
activities occur throughout all four phrases - in accordance with the CIPP framework previously detailed. 

Hypothetically speaking, however, a dynamic, process-oriented system that promotes high-level and 
multifaceted decision-making should impact both “functioning” and culture in significant ways. Gross impact can be 
measured quantitatively by comparing the SDSU/COE to other SDSU colleges or schools that are demographically 
and culturally/socially similar. Among the program data of value to collect are application, enrollment, graduation, 
and time -to -completion statistics; general student characteristics; class loads; and community placements. 
Quantitative data can also be collected via survey to measure attitudes/perceptions across a range of intensity 
scales. And, of course, system performance must be monitored (error types and frequency, nature and frequency of 
security breaches, technical team response time). The types of quantitative data envisioned call for both descriptive 
and inferential (primarily, nonparametric) tests of significance. 

Qualitative data may be gathered through semi -structured interviews or focus groups, and classroom 
observations may also be appropriate. Constant comparison, a well-respected technique for analyzing open-ended 
data drawn from multiple sources (Yin, 1994), will allow critical themes to emerge. Case studies or vignettes of several 
different types (exploratory, critical instance, program implementation, or cumulative) offer opportunities to focus on 
contextual comparisons across sites (Yin). 

Other mixed-methods processes are also wise to include. Content analyses may focus on course syllabi, 
selection of questions for inclusion on each semester’s comprehensive exam, midterm and final exams, and project 
types (individual or team). Action plans may allow for time series -style studies targeting data management, synthesis, 
packaging, and distribution. 

As a research endeavor, then, CAMS encourages school personnel (administrators, faculty, and students) 
to actively explore how tracking and reporting the nuances of “school business” can impact - perhaps even alter - 
the school environment in substantive ways. Such investigative goals are premised on several assumptions: 

• That a technology-based accountability system can help school personnel perform their assigned duties. 

• That communication within a school and externally with others is a primary function of a technology-based 
accountability system. 

• That data generated via the system are used in the context of multilevel decision-making. 

• That accountability systems differ in terms of type, functionality, and scope (Visscher, 1994). 

• That a successful accountability system must be customized to meet local needs. 

Understanding, from an evaluative perspective, how a sophisticated accountability system influences 
school operations, culture, and community relations allows us to identify areas for program revision/improvement. 

CAMS: Importance of Findings to Research and Practice 

There’s little value in instituting an accountability system if results (progress; system strengths and 
weaknesses; lessons learned) aren’t broadly disseminated. CAMS deployment begs for information sharing - at 
conferences, via peer-reviewed and practitioner-oriented journals , through demonstrations (online and traditionally 
facilitated). This project can build communities of practices focused on optimizing student performance and program 
responsiveness by showcasing: 

• the types of COE management activities or functions that system output can inform. 

• the ways in which the system impacts decision-making processes, as well as administrative and instructional 
roles and responsibilities. 

• how the system impacts communications. 

• how the system contributes to changed perceptions of information needs - specifically, from student-specific to 
comprehensive (e.g., departmental, college, or university-wide). 

• how automation affects the productivity of department and college personnel. 

• what educational benefits result from system implementation (e.g., positive changes in how instructional 
programs are planned, delivered, and assessed for effectiveness). 

• how the system affects instructor and/or administrator self-reflection. 
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• the challenges associated with system implementation, including creation of a shared vision and accommodation 
of cultural differences among constituencies. 

• the effects of differential access to data among system users. 

• how the system affects the way information is packaged and disseminated. 

So much is at stake . . . not the least of which is our graduates’ readiness for a fast-paced, technological 
world and team-based, highly -competitive work environments that call for incisive decision-making and problem- 
solving skills). In that light, then, such a research agenda cannot be ignored. 

References 

Abort, R. D., Wulff, D. H., Nyquist, J. D., Ropp, V. A., & Hess, C. W. (1990). Satisfaction with processes of collecting 
student opinions about instruction: The student perspective. Journal of Educational Psychology, 82{2), 
201-206. 

Acaro, J. S. (1995). The Baldrige Award for education: How to measure and document quality improvement. New 
York: St. Lucie Press. 

Anton, C. M., Dilla, B. L., &. Fultz, M. L. (1997, May 18-21). Assessing student attitudes: Computer versus pencil. 
Paper presented at the Annual Fomm of the Association for Institutional Research, Orlando, FL. (ERIC 
Document Reproduction Service No. ED 410 866) 

Bober, M. J. (2001). School information systems and their impact on school operations and culture. Journal of 

Research on Computing in Education, 33{4). Available online at: http://www.iste.Org/irte/33/5/bober.html 
Ramer, M. H., & Snowden, M. (1994). Using a management information system effectively for contract education 
programs. New Directions for Community Colleges, 85, 35-46. 

Rogers, E. M. (1995). Diffusion of innovations (4* ed.). New York: Eree Press. 

Stufflebeam, D. L. (2000). The CIPP model for evaluation. In D. L. Stufflebeam, G. E. Madaus, & T Kellaghan, T. (Eds.). 
Evaluation models: Viewpoints on educational and human services evaluation (2nd ed.). Boston: Kluwer 
Academic Publishers. 

Telem, M. (1996). MIS implementation in schools: A systems socio -technical framework. Computers & Education, 
27(2), 85-93. 

Telem, M. (1999). A case study of the impact of school administration computerization on the department head’s role. 

Journal of Research on Computing in Education, 31(4), 385-401. 

Telem, M., Sl Avidov, O. (1994). Management information system (MIS) impact on the loosely coupled nature of a 
high school: A case study. Planning and Changing, 25(3/4), 192-205. 

Worthen, B. R., Sanders, J. R., &. Eitzpatrick, J. L. (1997). Program evaluation: Alternative approaches and practical 
guidelines (2nd ed.). New York: Longman. 

Visscher, A. J. (1994). A fundamental methodology for designing management information systems for schools. 

Journal of Research on Computing in Education, 27(2), 231-249. 

Yin, R. K. (1994). Case study research: Design and methods (2"“* ed.). Thousand Oaks, CA: SAGE Publications. 



104 



Instructional Technologies in Developing Countries: A Contextual Analysis 

Approach 



Sonia Arias 

Education Development Center, Inc. 

Kevin A. Clark 
George Mason University 



Abstract 

This paper describes the problems encountered when working with instructional technologies in developing 
countries. These problems include socio-cultural and environmental factors such as lack of human capacity and 
poor infrastructure. The authors argue that the traditional instructional analysis phase of the instructional design 
process is not designed to catch this type of challenge and can therefore lead to failed instructional technology 
interventions in developing countries. The authors argue for a broader, context-based definition and deployment 
of the analysis phase. The authors describe the Tessmer and Richey (1997) context-based analysis model and 
propose specific areas of adaptation to the model. The authors argue that the adapted Tessmer and Richey 
context-based model presented in this paper can better anticipate the challenges likely to be found in developing 
country contexts and therefore better prevent failed IT interventions from occurring in developing countries. 



Introduction 

Countries throughout the developing world are struggling to bring their populations out of poverty. For the 
past several decades international development efforts have focused largely on economic, infrastructure, and social 
reform policies. However, with the advent of the Internet and its profound impact on society, commerce, and 
education, development practitioners are compelled to examine the role of information communication technologies 
(ICTs) on development. Just as ICTs have catapulted industrialized countries into the Information Age, many 
countries in Africa, Asia, and Latin America are anticipating that ICTs can leapfrog several stages of development 
and thus bring wealth and improved social benefits in governance, health, and education. 

In the area of education and training, developing countries are faced with monumental problems. World 
population growth is such that by 2025 about 100 million children worldwide will be without schooling (Crede & 
Mansell, 1998). The implications of this on the demand of teacher training for example is equally monumental. 
Ministries of education throughout the southern hemisphere are increasingly looking at the scalability of ICT -based 
distance learning options to meet this challenge. Additionally, governments are justifying the implementation of 
instructional technology initiatives within primary and secondary schools on the basis that: 

a) Developing economies need populations skilled in information technologies; 

b) All citizens need to be able to take advantage of technological developments as they become avaOable; 

c) Technology can enhance existing curricula; 

d) Technology can make curricula more relevant and redefine the teachers’ roles; and 

e) The Internet allows students and education practitioners to communicate with others (Perraton, 2000). 
However, designing, developing, and implementing instructional technology initiatives in a developing country is by 
no means a simple endeavor. Issues of poor connectivity, lack of human capacity, scarcity of appropriate content, 
and ever diminishing budgets are dealt with on a regular basis. 

In spite of this, education practitioners in developing countries are moving forward and implementing 
instructional technology initiatives in order to improve access to education, to increase the quality of education, or to 
implement educational reform (Perraton, 2000). These initiatives often take the form of incorporating computers into 
elementary and secondary schools, or developing synchronous and asynchronous distance learning systems for 
secondary, tertiary, and other adult training environments. 
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There is an increased application of instructional technology in developing countries, but because of the 
increasingly scarce supporting resources it is imperative that effective and meaningful instructional systems design 
(ISD) principles continue be included in the design of new technology-based learning environments. However, 
“every effort should be made to avoid simple replication of the ICT -based learning processes used in industrialized 
countries.’’ (Crede & Mansell, 1998). Indeed, when applying instructional systems design to an instructional 
technology initiative in a developing country it is important to keep in mind that the design process never takes place 
in a vacuum. In fact, the instructional design process often occurs within a highly volatile socio-economic, cultural 
and political context. Some researchers state that failure to analyze the broad context surrounding the instructional 
event will inevitably lead to countless failed instructional design initiatives in developing countries (Romiszowski, 
1981, 1989; Weston 1989). 

Tessmer and Richey (1997) underscore the importance of analyzing context in the instructional design 
process by developing a context -based analysis model. In essence, this model calls fora broader definition of the 
analysis phase and a concomitant inclusion of factors not included in the traditional analysis phase. These 
additional factors include learner, environmental, and organizational factors that have an impact before, during, and 
after the instructional event. 

This paper contributes to the existing body of literature (Romiszowski, 1981,1989; Weston 1989; Tessmer & 
Harris, 1992) by presenting an adaptation of Tessmer and Richey’s (1997) context -based analysis model. This 
adaptation is necessary because the increased level of instructional technology initiatives taking place in developing 
countries requires that instructional designers be appropriately prepared to work with the unique challenges found in 
developing country environments. Tessmer and Richey’s model, though it broadens the analysis phase, is still 
insufficient for a developing country context. Thus, the adapted model presented in this paper calls for the front-end 
analysis phase of the design process to go beyond the traditional instructional needs assessment approach and to 
broaden Tessmer and Richey’s analysis of the socio -cultural, environmental, and organizational factors surrounding 
the instructional problem. It is hoped that this adapted model, when applied, will increase the success rate of 
instructional technology initiatives in developing countries. 

First, the authors will outline some of the ways in which instructional technologies are being deployed in 
developing countries. Second, the commonly known challenges of instructional technology initiatives in developing 
countries will also be outlined. Third, based on the challenges described, the Tessmer and Richey model (1997) 
model will be adapted to include contextual factors specific to developing countries . 

Instructional Technologies in Developing Countries 

Instructional technologies in varying forms of delivery and formats such as radio, television, personal 
computers, the internet, print, cassettes, and CD-ROMs are being deployed throughout the developing world in 
an effort to meet the critical educational problems of numbers, resources, and quality at all levels. 

In the area of distance learning for example, radio has been in use at the primary education level in Australia 
since the 1950’s. In the 1970’s, interactive radio instruction was launched in Nicaragua in order to teach mathematics 
to isolated children living in rural areas (Mayo, 1999). At the secondary level, Mexico’s Telesecundaria program has 
been providing televised instruction to seventh, eighth, and ninth grade children who cannot get to conventional 
schools because of their remote living conditions. This successful program has been in operation for over thirty 
years (Calderoni, 1998). At the tertiary level, China in 1990, reported that its television universities had enrolled 1.83 
million students. In the area of teacher training, Zimbabwe and Tanzania have deployed instructional technologies in 
the form of radio, and print to improve the quality of teachers’ performance in the classroom (Perraton, 2000). Finally, 
the use of computers in schools has been argued for and tested in several developing countries. ChiXt' & Enlaces 
Network was launched in 1993 to “. . . provide teachers and students with new and improved instructional content 
and methods, increased information resources for research and analysis, and improved communications for 
collaboration and dissemination of ideas (p.5, Potashnik, 1996). 

These examples underscore the fact that the developing world has indeed turned to instructional 
technology, whether it is for distance learning, training, or for the classroom. This has an impact on instructional 
design since all of these educational applications of technology are explicitly or implicitly being designed using the 
instmctional systems design process. The following section describes the types of challenges often encountered 
when implementing instructional technology initiatives in developing countries. However, these are challenges that 
the traditional analysis phase of the instructional systems design process is not necessarily designed to solve. 
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The Challenges of Implementing Instioictional Technology in Developing Countries 



Instructional technology initiatives in developing countries can take on several forms. Because English is the 
predominant language in the technology arena, projects may consist of creating language-specific, culturally 
appropriate content to be distributed via the Web or CD-ROM. Other interventions might include improving the 
quality of classroom instruction, providing vocational or technical training, or teaching English. The challenges of 
implementing instructional technology interventions of this type in developing countries span the social, cultural, 
environmental, and institutional contexts. An illustrative list of these challenges follows. 

Socio -cultural and Human Capacity Eactors 

Incipient Culture of Participation: A culture of participation, information sharing, and open discussion should not be 
assumed to exist in all developing nations. This implies that a “community of practice’’ for example, may need to be 
based on a very thorough analysis of the degree to which users are actually willing to share information on a routine 
basis. Additionally, institutional change will most often involve a ‘top-down’ process and thus commitment to 
change or adopting new procedures may present some challenges. 

Centralized Decision-Making.- Designers will find that many decisions are made by a potentially unstable, central 
authority that may or may not take into account the needs and desires of local constituents. The implications of this 
are that instructional designers will need to go out of their way to ensure that users have the opportunity to provide 
input and feedback into the design process. 

Resources for Learning and Support: If the instructional technology being designed requires users to undergo a 
learning curve, motivation, release time, and compensation must be addressed during the design phase. Eailure to do 
so may lead to a design or implementation strategy that is not appropriate for users. 

Lack of Budget Planning: It is possible that institutional capacity will have limited budget planning capabilities. This 
contextual factor could have a negative impact on the implementation of new technology if maintenance, support, 
and upgrades are not included in the budget. 

Scarcity of Qualified Personnel: Qualified staff required to maintain the new instructional technology may not be 
available. This implies that the design process for developing country contexts needs to consider the amount of 
human capacity available for maintenance or ensure that the design incorporates a very simple maintenance 
structure. 

Infrastructure Eactors 

Unreliable Power and Telecommunications Grids: Users in developing countries may only have access to the internet 
through low bandwidth connections. Designers should therefore aim to design systems that accommodate low 
bandwidth limitations by using fewer graphics and media. 

Hardware, Software, Maintenance and Support: It is possible that computer hardware and equipment will need to be 
imported in order to support the new system. When this is the case, it will be necessary to budget for high import 
duties and ensure that appropriate maintenance, support, and upgrades can be secured locally or with the overseas 
vendor. Eailure to take this into account in the analysis phase could lead to the design of unsustainable systems. 
Adequate Eacilities: The climate in many developing countries is extremely hot and humid. System design needs to 
ensure that adequate cooling, ventilation, and dust control will be available. 

A close examination of these challenges leads one to conclude that many of them are by no means strictly 
instructional in nature and one cannot assume that they will be accounted for in the traditional analysis phase of the 
instructional system design process. And yet these challenges have the potential to positively or negatively impact 
instruction as well as instructional technology initiatives in general. In other words, depending on local contexts and 
circumstances, social, cultural, environmental, and institutional factors can make or break instructional technology 
initiatives in developing countries. It is therefore imperative that the instructional design process be much broader, 
and employ a context -based analysis approach when designing instructional technologies for developing countries. 

A Context- Based Analysis Approach 
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Instructional systems design can be defined as the application of systems theory, information theory, and 
cognitive psychology to the process of designing effective and efficient instruction. Smith and Ragan (1999) define 
instructional design as the . . systematic and reflective process of translating principles of learning and instruction 
into plans of instructional materials, activities, information sources, and evaluation” (p. 2). This process of design 
usually consists of five key components: Analysis, Design, Development, Implementation, and Evaluation. The 
ensuing acronym, “ADDIE”, is often used to describe this approach to instructional design and is thus commonly 
referred to as the ADDIE model. The analysis phase concentrates on examining three basic components: the 
instructional context, the prospective learners, and the learning task. Weston (1989) points to the key importance of 
a ‘front-end’ analysis for developing country instructional technology interventions. However, as already 
mentioned, this type of analysis is not designed to analyze the broader context. 

Tessmer and Richey (1997) define context as a ‘‘multilevel body of factors in which learning and performance 
are embedded.” The contextual analysis is a model that investigates the influence of context on three aspects of 
learning; orienting, instructional, and transfer. The orienting context occurs before the learning event and deals with 
factors that influence student motivation and cognitive preparation. The instructional context focuses on factors 
that are present during the learning event such as the physical, social, and symbolic resources. The transfer context 
deals with the environment where the learning is applied after the learning event. Eor each of these temporal 
contexts; orienting, instructional, and transfer, there are three embed contextual levels. These levels include the 
learner, environment, and the organization. These contextual levels may contain several factors such as learner 
profile, social support, or incentives. Table 1 summarizes Tessmer and Richey’s (1997) context -based analysis model. 
This model is a great start, but does not include many of the factors and issues that learners and instructional 
designers in developing countries deal with on a daily basis. 



Table 1: Context -based analysis model (Tessmer and Richey, 1997) 





Orienting 


Instructional 


Transfer 


Learner Factors 


Learner profile 
Goal Setting 
Perceived utility 
Perceived accountability 


Learner role perception 
Learner task perception 


Utility percept. 

Perceived resource 
Transfer coping strategy 
Experiential background 


Immediate 

Environment 

Factors 


Social Support 


Sensory conditions 
Seating 

Instructor role perception 
Learning schedule 
Content culture 


Transfer opportunity 
Social support 
Situational cues 


Organizational 

Factors 


Incentives 
Learning culture 


Rewards & values 
Learning supports 
Teaching supports 


Transfer culture 
Incentives 



In an effort to apply the contextual analysis model to learning situations in developing countries, the 
original Tessmer and Richey (1997) model has been expanded to included additional environmental and 
organizational factors in the orienting context. The additional immediate environment factors are motivation, release 
time, compensation, participation culture, availability of electricity, and appropriate facilities. All of these factors 
relate to supporting the learner prior to the learning event. Learners must be motivated, be given the time as well as 
the compensation to participate in the learning event. Additionally, the culture of participation needs to be taken into 
consideration. Are people likely to participate or is it against their cultural or social norms? Lastly, the availability of 
basic resources like electricity and facilities should not be overlooked. 

The following organizational factors were added to the orienting context. These additional factors included: 
centralized decision making, budget planning capacity, and human capacity building. In many developing countries, 
learning systems and instructional technology interventions can not be created or implemented without involvement 
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or approval from other organizations. Having a centralized decision making process can hamper the process of 
obtaining by-in and cooperation from all involved parties. The notion of budget planning and human capacity 
building are both related to the ability of the developing nation to create the necessary resources for budgeting, 
support, and maintenance. In addition to the learning event, organizations may also need to assist people in the 
acquisition of technical, accounting, and managerial skills. 

This expanded context model focuses on specific issues that may be related to the design, development, and 
implementation of instructional technology interventions in developing countries. 



Table 2: Adapted Context -based analysis model; based on Tessmer and Richey (1997) model 





Orienting 


Instructional 


Transfer 


Learner Factors 


Learner profile 
Goal Setting 
Perceived utility 
Perceived accountability 


Learner role perception 
Learner task perception 


Utility percept. 

Perceived resource 
Transfer coping strategy 
Experiential background 


Immediate 

Environment 

Factors 


Social Support 

Motivation 
Release time 
Compensation 
Participation culture 
Availability of electricity 
Appropriate facilities 


Sensory conditions 
Seating 

Instructor role perception 
Learning schedule 
Content culture 


Transfer opportunity 
Social support 
Situational cues 


Organizational 

Factors 


Incentives 
Learning culture 
Centralized decision making 
Budget planning capacity 
Human capacity building 


Rewards Sl values 
Learning supports 
Teaching supports 


Transfer culture 
Incentives 



Conclusions 

This paper has presented an adapted context -based analysis approach for implementing instructional 
technology in developing countries by calling for the front-end analysis phase of the design process to go beyond 
the traditional instructional needs assessment approach and having it include an analysis of the socio -cultural, 
environmental, and institutional factors surrounding the instructional problem. Given that there is an increased 
application of instmctional technology in developing countries and limited human capacity to appropriately design 
instructional technology interventions, it is imperative that instructional designers working in developing countries 
take into account the contextual factors outlined in the adapted Tessmer and Richey (1997) model. This adapted 
context -based analysis model for instructional design if appropriately applied, may increase the probability of 
successful instructional technology initiatives in developing countries. Although this context -based model was 
expanded to include factors related to developing countries, the factors described in this paper exist in developed 
country settings as well. The authors would therefore encourage the use of this expanded model in settings that can 
include but are not limited to certain rural and urban areas in industrialized countries as well. 
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Managing Online Mega Classes 

Tena F. McQueen 

Robert A. Fleek, Jr. 

Columbus State University 
Background 

Columbus State University (CSU) is a unit of the University System of Georgia and offers graduate and 
undergraduate degrees to over 6000 students. Many CSU students work full- or part-time and come to CSU with a 
great range of computer competencies. One of CU’s institutional values is computer literacy. 

During a self-assessment it was determined that some students had found a curriculum path that allowed 
them to avoid or postpone courses with computer content until their senior year, thus defeating the concept of using 
computer skills in other courses. It was therefore decided to require a literacy course of all entering freshman. All 
entering freshmen would be automatically enrolled in the new literacy course and strongly counseled against 
dropping the course. It was estimated that in excess of 800 students would be enrolled. 

Faculty Issues 

With the anticipated enrollment of 800 students in computer literacy classes, where normally there were 150 
enrolled, faculty resources would be stretched. Since it was expected that students enrolled in the literacy course 
would not decrease the number of hours in other classes, there was little opportunity to shift resources or teaching 
assignments. Developing creative solutions for this increased credit hour production became essential. 

Asynchronous Courses 

The authors had experience in developing solutions to the changing needs of the student population. One 
solution was the use of a CD-ROM tutorial package. The authors had also developed a complete Internet course for 
delivering computer skills. 

Results from these initial asynchronous courses showed a high degree of success and acceptance by 
students and administration. The courses offered reduced seat time, required fewer physical facilities, and met the 
needs of commuter students. These courses were not without their flaws and in retrospect resource savings to 
students and the institution were at the expense of instructor time. 

Proposed Solution 

The institution decided that a combination of CD-ROM and WebCT materials would be used to deliver 
course content. Management of the course would be vested in one instructor responsible for designing content and 
assigning grades. Teaching and student assistants would support this individual. Students were required to 
purchase the CD and were given written directions on how to access online course materials. As with all great plans, 
some problems only appear during implementation. 

Student Misconceptions 

Online course content is often viewed as less rigorous, less time consuming, and therefore, more 
“manageable.” Our experience indicates that students underestimate their off-campus hardware/software resources 
and often ignore minimum computer requirements essential for success. Misjudging their ability to manage time, set 
goals, and to navigate Web-based material was another impediment. 

Institutional Misconceptions 

One misconception is that online courses give faculty more free time. Some think that less class contact time 
requires less effort and underrate the complexity of delivering a course online. They may therefore argue that 
hundreds of students be permitted to enroll in such courses. 

infrastmeture Considerations 
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Since this course was required of all entering freshmen, a mechanism was developed for automatically 
enrolling students during registration. University Enrollment Services developed procedures, policies and fee 
structure for this class. A computer lab facility was staffed and available six days a week to assist students. 
Anticipating the ebb and flow of student computer lab use became one of the serious management challenges. 

Instructional Technology Services ordered equipment which would permit creating large numbers of CD- 
ROMs and printing labels for each. 

Coordination with the Registrar’s Office was a key element in notifying students of course requirements and 
logis tics. Information appeared in the printed Schedule Booklet, in the online version on the University’s Web site 
and announced at each Freshman Orientation Session. 

The university’s center for Computer Information and Networking Services prepared for increased student 
activity in the university’s open lab by hiring and training additional student assistants. A “Help” line was 
established to answer student questions via phone, e-mail, or face-to-face. The Help Line addressed network matters 
such as logon and password problems, but most often could not provide assistance with Web, course-content 
problems. 

Instructor Considerations 

While many instructors can develop the materials needed for an asynchronous course, utilizing materials 
developed by publishers or software companies is more efficient. Finding Web-based materials that match course 
goals and objectives is the challenge. If there is a short period of time between developing the asynchronous course 
and delivery, requisite time to develop course materials becomes a key factor. 

Publishing as much course information as possible to a public Web site allows unobstructed access. This 
can be an important consideration for students just learning to navigate the Web and often view multi-logons as 
obstacles. Course content can be placed on a password protected site so only those registered students can access 
course content, postings, quizzes, etc. 

Course management is crucial in all classes, especially in over-sized classes. Web-based, course 
management tools can organize instructor announcements, student comments, maintain grades, display calendars, 
deliver content/quizzes and allow students to submit assignments. Another question is “Who will provide technical 
support for students accessing Web-based material?” 

Utilizing teaching and or student assistants adds another facet to the online course scenario. Qualified 
teaching/student assistants must be recruited and trained to deal with issues presented to them by students in 
asynchronous classes. As the number of assistants increases, the more the role of the instructor shifts to “manager.” 
In large, over-sized classes, the instructor begins shifting from a content expert to a manager of content and services. 
Large classes rely on many other support facilities including computer labs, teaching assistants, and others. As more 
resources are required, the span and depth of management control expands. Adding additional levels or numbers of 
support personnel does not always solve the problem since the instructor must now manage more people and 
becomes further removed from the classroom. For some academics, this may be an unsatisfactory result. 

Similarly, testing in online courses often shifts from testing a student’s knowledge of facts and or skills to 
grading students on their completion of certain activities. For example, how many times they posted a response to the 
discussion topic of the week. Testing online concerns include credentialing, delivering a skills -based exam, and 
evaluating assignments given to students outside a proctored environment. 

Conclusions 

Clearly State the Objectives: The over-sized course suffered from many poorly understood issues. The 
course was originally established to provide a vehicle for mass coverage of computer literacy. As the size of the 
project increased, other objectives became predominant. 

Student Maturity: Not all students are mature enough to succeed in a self-paced course. An analysis of the 
post-class student survey indicated that many students had the initial perception that the class would somehow be 
easier or less rigorous than other classes. Many students misjudged their ability to complete the course work in the 
amount of time allotted. 

Planning: Resource planning and student retention are key issues in managing super-sized classes. While it 
may be possible to control these classes with qualified graduate students, some level of student screening and 
counseling must be in place for self-paced mega classes. 
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Management: Instructors of over-sized classes soon find that they have become managers of the classroom 
experience rather than deliverers of content. What is a worrisome task in a class of 30 becomes a Herculean task in a 
class of 800. Merely getting a consistent message to 800 students can be almost impossible. 
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A Design Model for the Internet-Based Electronic Performance Support Systems 



Hakan Tuzun, Hasan Cakir 
Indiana University Bloomington 
Abstract 

This presentation describes the design and development of an Internet-Based Electronic Performance Support 
System called the Systemic Change EPSS. The Systemic Change EPSS provides systemic change facilitators with 
the information, tools, and resources they need, and accessibility to other facilitators who are dealing with 
systemic change effort. This presentation outlines design issues for Internet-based EPSSs and strategies for 
developing such systems. Based on the formative research experience, we propose a design model for Internet- 
based EPSSs. 



Introduction 



What is EPSS? 

There are different definitions of Electronic Performance Support Systems (EPSS). Gery (1995), who coined 
the term, defines an EPSS as a tool for just-in time performance support in the work place. In another term, an EPSS is 
a complete system or integration of performance support tools to achieve certain tasks in the work setting. The goal 
of an EPSS is “to provide whatever is necessary to generate performance at the moment of the need’’. 

Similar to the case of different definitions of EPSSs, there are also different ideas about the components of 
an EPSS (Cagiltay, 2002). In its basic form an EPSS has the content, the support system, and the user interface that 
acts as an umbrella for the first two components (Gery, 1995). Cagiltay (2002) summarized different component views 
from Schwen et al. (1993), Hannafin et al. (2000), and Barker and Banerji (1995). All these component views contained 
the basic view Gery (1995) put forward. 

EPSS Design Models 

A small-scale EPSS system might be designed with little or no planning (for example, see the case in Hoyt, 
Stockman & Thalmann, 1997). However, the use of this approach for the larger systems may not be appropriate since 
it might cost more time and money. Therefore, we need a systematic planning approach to design such systems, 
which is called a model. Gustafson and Maribe (1997) explain the role of model as “a model is a simple representation 
of more complex forms, processes and functions of physical phenomena or ideas. Models are constructed to 
conceptualize representations of reality.’’ 

The purpose of an Instructional Design (ID) model is “to convey key concepts and processes to be 
included in a particular approach’’ (Molenda et al., 1996). Models tell us the critical success factors to instructional 
design. To be more specific, an ID model tells us what to do and when to do it, and it barely tells how to do it. 
Molenda et al. (1996) point to the importance of an ID model by this example: “. . .experience leads the expert away 
from the cookbook and toward improvisation. But for the apprentice chef, the cookbook is the vital link to 
maintaining quality and consistency from day to day.’’ The ID model is the cookbook that will address these 
concerns for instructional design. 

In its early times there was not any specific research-based models available for developing an EPSS (Scales 
& Yang, 1993). The researchers indicated that it was essential to have an experienced project manager observe the 
EPSS development process. Since then several EPSS design models have emerged. Pour of these models are 
explained below briefly. As Love and March (1975) expresses, the same thing might have several different models. 

Law (1995) organized EPSS design issues into 3 main categories. These categories are front-end analysis, 
development, and evaluation and research. He presents an EPSS case study, which was designed based on his 
model. His model includes principals from instructional systems, software engineering, performance technology, and 
formative evaluation. He concluded that a variety of design strategies should be examined to find out the best 
solution for the EPSS situation. 

The Des Jardins-Davis EPSS design model is a linear model, which has 3 phases (Des Jardins & Davis, 

1995). Phase 1, marketing, consists of 1 step: marketing the EPSS. Phase 2, investigating, consists of 3 steps: planning 
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the EPSS, conducting specification analysis of the EPSS, and presenting findings. Phase 3, doing, consists of 6 steps: 
developing a maintenance strategy, preparing EPSS development plan, building the EPSS, doing the implementation, 
doing the evaluation, and maintaining the EPSS. Their EPSS design model lacks methodology. In addition, there was 
no application of the model to any specific EPSS case. 

Lohr (1998) indicated that the design and development skills required to produce an EPSS fall into many 
domains. She offered ADDIE model as an easy to implement design strategy, until easier EPSS development tools for 
the instructional designers are developed. She describes how the model was utilized to create a graphical user 
interface, in the form of an EPSS, for a university computer course. 

Sheu (2000) followed a socio -technical design framework, user-centered perspective, and a rapid 
prototyping approach when she designed an EPSS for doctoral students taking their qualifying exams. She 
developed the model by looking at the previous work practices of the time and psychological behaviors of users 
preparing to take the qualifying exams. 

In common with all these 4 design models mentioned above is the use of ADDIE model, which is a generic, 
systematic, and classical approach to instructional design. The word ADDIE represents Analysis, Design, 
Development, Implementation, and Evaluation. In some of these cases, designers utilized a modified version of the 
ADDIE model. Actually Kmse (2002) states that “there are more than 100 different ID models, but almost all are based 
on the ADDIE model”. 

Internet-based EPSSs 

The literature shows that EPSS is still an immature field lacking a generally accepted design model, let alone 
models for Internet-based EPSSs. 

Using the Internet for different purposes has entered a revolution during last two decades. Especially, after 
the development of the hyperlink on the World Wide Web (WWW), the Internet has offered more user-friendly 
environments (Starr, 1997). While organizations and educational institutions have been using the Internet to 
distribute information for a long time, the implications for the instmctional design and delivery of EPSSs over the 
Internet have not been sufficiently explored. Laffey and Musser (1996) envisioned Internet-based tools as a new form 
of EPSS, creating new performance spaces. They indicated the properties of a dynamic support system as the ability 
to change with experience, to be updated, and adjusted by the performer, and augment other resources found in the 
user’s community. However, they did not explain how they merged the Internet and their EPSS, which was built to 
support pre -service teachers, field-based mentors, and college faculty as they collaborate, engage in practice, 
document their efforts, share their experiences, and assess outcomes. 

Research Questions 

The purpose of this study is to propose a design and development model for Internet-based EPSSs. Eor this 
purpose the following research question was investigated: 

• What should an effective, efficient, and appealing design model incorporate for the design and 
development of an Internet-based EPSS? 

Systemic Change EPSS 

The idea of creating an EPSS for systemic change was born after a discussion between an Indiana 
University graduate student and a professor at the same university. After deciding on designing an Internet-based 
EPSS for systemic change facilitators, the designers started to search for a model that they could follow throughout 
the design process. However, the literature had very limited number of design models for the EPSS design. Due to 
lack of Internet-based EPSS design models, designers decided to create their own model by using the Internet-based 
EPSS design case. This way they also added new information to the body of knowledge in our field and they 
provided a reflection of their design process. The model explained here is based on a design process of the Internet- 
based EPSS case. 

Two graduate students and one professor worked on designing the Internet-based EPSS. The designers 
followed a formative research methodology to create the model. The activities to design the Internet-based EPSS 
have provided the data for creating the design model. 
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Methodology 

The model developed in this study is a design theory. Theories in applied fields like instructional design can 
be categorized as design theories and descriptive theories (Reigeluth, 1999). Design theories are different from 
descriptive theories. While descriptive theories explain existing situations or phenomenon like learning theories, 
design theories provide models and guidelines for designing and/or improving instructional practices (Reigeluth, 
1999). 

Formative research is an action research to develop or to improve design theories for designing instructional 
practices and processes (Reigeluth, 1999). Formative research is driven by “what worked well”, “what did not work 
well” and “what improvements can be made” questions. A practice developed by following a theory may show the 
weaknesses of the process, which develops the theory. By examining the case, the theory may be improved for 
further applications. Research theories are based on some values (Reigeluth, 1999). Descriptive theories emphasize 
the validity of the research since the answer to “what is” question matters, whereas design theories value the 
preferability, thus the answer to “what is better” question has more importance. Formative research values the 
process. Reigeluth and Frick (1999) categorized the values of design theories in three categories: effectiveness, 
efficiency and appeal. Effectiveness implies to get to the target or to get the job done correctly. Efficiency is to reach 
to the results by using optimum resources. Appeal is the degree the process is enjoyable for the designers and the 
users. 

Reigeluth and Erick (1999) developed three methodological procedures for formative research: designed 
case, in vivo naturalistic case, and post facto naturalistic case. Each procedure can be used for an existent theory or 
for a new theory. In our case, a model for designing Internet-based EPSS, we took the approach of designed case for 
a new theory. The steps involved in this methodology are, with time base sequence: 

• Creating a case to generate design theory 

• Collecting and analyzing formative data on the instance 

• Revising the instance 

• Repeating the data collection and revision cycle 

• Eully developing the tentative theory 

We applied these steps to develop the model for designing an Internet-based EPSS. Since the created case 
is a work in progress, so is the model. Constant revis ion cycle takes place in the development of the product and in 
the process to produce it. 

Proposed Design Model 

The model presented here is a tentative model for designing an Internet-based EPSS (Eigure I.). The 
evaluation and modification of the model take place throughout the design process. The model consists of 10 steps. 
Each step has sub steps for explaining “how to do” for implementing the main step. Sub steps of the model contain 
procedures specialized for the Internet-based system design. 
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The model is an enhanced application of the ADDIE model. The difference from the ADDIE model is that the 
model has two design steps (step 2 and step 6) and two decision points (step 3 and step 5) before taking further 
steps towards building the system. Since the available tools and data communication bandwidth are limited on the 
Internet medium, the designers have to consider these variables in their design decisions. Because of these 
limitations, the designers have to consult with the client and get the approval of the client before building any part of 
the system. 

The Internet allows the designers to disseminate the knowledge to a broad audience with easy and low 
update costs. In order to maintain and to update the EPSS with the most current information, a maintenance strategy 
should be designed. After compiling all the data about design plans, maintenance, and total and on-going costs of 
the system, the designers should meet with the client to decide whether the system should be developed or not. 

Components of the Model 

1. Analysis 

1.1. Organization’s or individuals’ needs (client’s needs): What performance will be increased. How and in 
which direction performance will be increased. 

1.2. User analysis: Demographics and computer literacy level of the potential users. 

1.3. Scope analysis: Content analysis; according to the needs of the client what content will be covered. Task 
analysis; tasks involved in the content will be analyzed. 

1.4. Technical analysis: Eeasibility - Connection speeds - Internet Service Provider (ISP) software support 
capability - Browsers that are going to be used - Site address 

2. Pre -Design 

2.1. Set-up the specific objectives for the EPSS; what it is going to be accomplished. 

2.2. Determine functional specifications according to the needs. 

2.3. Determine forms of the functions. 

2.4. Determine the connections between the functions based on task analysis. By visiting the task analysis 
these connections can be determined. 

2.5. Chunk the information. With the help from content creator or subject matter expert, chunk the information 
based on five plus or minus two rule. 

2.6. Determine connections between information chunks. After determining the information chunks, determine 
the connections between them if there are. 

2.7. Determine the components of the EPSS as performing technical functions. What components serve as 
technical functions of the system. 

2.8. Perform feasibility of the functions. Determine if the functions can be implemented according to technical 
analysis and user analysis. 

2.9. Provide alternatives for the functions. 

2.10. Revise compatibility of functions and components. 

2.1 1. Determine the responsibilities of the client in the design process. 

3. Pre -Decision Point: In this step, client and the development team meet in order to go further or to terminate the 
project based on the collected data and the resources required to develop the system. 

3.1. Prepare a general outline of the EPSS to the client. 

3.2. Discuss the functions with the client. 

3.3. Discuss the alternatives of the functions. 

3.4. Discuss the possible costs with the client. 

3.5. Discuss the responsibilities of the client. 

3.6. Make DO or DO NOT DO decision. 

4. Develop a maintenance strategy 

4.1. Identify on-going maintenance requirements. 

4.2. Develop skill/personal requirements. 

4.3. Estimate cost of maintenance as one time cost and continuous costs. 

5. Decision Point 



118 




5.1. Discuss with the client total system and maintenance costs, procedures. 

5.2. Get approval of the client before starting the development of the EPSS. 

6. Build EPSS 

6.1. Prepare a big picture / outline of the system 

6.1.1. Components 

6.1.2. Connections 

6.2. Determine content of each element / component. 

6.3. Select appropriate software for each component and its function. 

6.4. Manipulate the content according to data flow speed. 

6.5. Determine file name hierarchy in the site. 

6.6. Determine general rules for interface design to assure consistency throughout the product. 

6.7. Assign the tasks to the teams. 

6.8. Integrate the elements. 

7. Conduct usability tests: Usability tests should be conducted on potential users of the system to fix or eliminate 
problematic parts. 

8. Implement the product on the Internet Service Provider (ISP) server. Install the system to ISP server. In order to 
protect the product from outside threats, appropriate security settings should be configured. 

9. Get feedback from authentic users and do the revisions as needed. After implementing the EPSS, authentic users 
(in our case systemic change facilitators) send their comments and feedback about the product. According to the 
change requirements, the designers should change the parts as necessary. 

10. Launch the maintenance plan and disengage with the client. After the design of the system is finished, in order 
to keep the site updated and running, maintenance plan should be launched. Depending on the complexity of the 
system and the skill level of the maintenance personnel, the designers should make decisions about the support 
level they will provide and how the maintenance personnel training will be accomplished. 

Limitations of the Model 

The presented model here has a narrow scope. It is for the design of Internet-based EPSSs. Although it was 
initially generated starting with the generic ADDIE model, it is different from ADDIE model and it may not be 
generalized or utilized in instructional design and development processes. The model does not have a great flexibility. 
Since the details make a difference, all of the steps of the model should be implemented in the design process. When 
using the model, the design team needs to collect scrutinized data. This requires a lot of time for front-end analysis. 

Conclusion 

We applied a formative research approach to the EPSS design. A review of the literature on EPSS design 
models indicated that ADDIE model was largely utilized for the design of the EPSSs. Our model is also an enhanced 
version of the ADDIE model. The difference from ADDIE model is that the model has two design steps and two 
decision points before taking further steps. In addition, we provide detailed guidelines on how to implement certain 
steps. With the help of our case, a systemic change EPSS, we are still in the process of developing this model. 
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An Internet-Based Electronic Performance Support System for Systemic Change in 

K-12 Settings 

Hasan Cakir, Hakan Tuzun, Charles M. Reigeluth 
Indiana University Bloomington 
Abstract 

We present an Internet-based Electronic Performance Support System (EPSS) called the Systemic Change 
EPSS. Systemic change facilitators can use this system to access the tools, resources, and information they need in 
the systemic change process. A general outline of the Systemic Change EPSS is introduced. Components of the 
Systemic Change EPSS and their interconnections are also explained, along with field test and usability test data. 

Introduction 

The purpose of systemic change is to create a better educational system than what currently exists. 

Systemic change is a collective effort that should be fostered by school, community and student populations. Yet, 
the efforts of these people and organizations may not be effective or efficient without receiving enough guidance. 
Guidelines for these change efforts can be provided by experienced facilitators and researchers in the systemic 
change field. Jenlink, Reigeluth, Carr, and Nelson (1996) developed a systemic change guidebook to help facilitators 
create and sustain systemic change efforts. 

The guidebook consists of 26 discrete events that occur at distinct points during the change effort. In 
addition to this, there are 18 continuous events that should be engaged in throughout the change process. Each 
discrete and continuous event is made up of several activities. Each of these activities is comprised of about 10 
pages of detailed guidelines, considerations, and tools. Since the guidebook is so thick and heavy, it might not be 
practical to carry it to many meetings conducted during the systemic change effort. It would also be expensive to 
copy it for key participants’ use. Eor these reasons, the authors of the guidebook decided to produce it in the form of 
an Electronic Performance Support System (EPSS), using hypermedia and the Internet. A team of designers and 
developers volunteered to accomplish this project. 

An EPSS is a tool for just-in time performance support in the work place. An EPSS is a complete system or 
integration of performance support tools to achieve certain tasks in work settings. The goal of an EPSS is “to provide 
whatever is necessary to generate performance at the moment of the need’’ (Gery, 1995). Since the computer networks 
and especially the Internet have become ubiquitous, we have now opportunities to implement EPSSs which are 
universally available on demand at any time and any place. 

The goal of the Internet-based Systemic Change EPSS is to provide the systemic change facilitators the 
tools, resources, and information they need in the process, and accessibility to the other facilitators and communities 
who are dealing with systemic change. The end product offers more functions than just being merely an electronic 
version of the guidebook. 

Functional Specifications of the Systentie Change EPSS 

A well planned needs analysis can be used as an information-gathering tool by a wide range of 
organizations under a variety of conditions (Reviere et al., 1996). A qualitative development methodology using semi- 
structured interviews was used to address the needs of current and potential users of the systemic change EPSS. 

Two systemic change facilitators were interviewed to figure out the functional specifications of the systemic 
change EPSS. Eacilitators were selected with purposeful sampling. At the time of the interviews, the facilitators were 
dealing with systemic change efforts at an urban school district in Indianapolis, IN. Interviews took approximately an 
hour long and were audio recorded. The interviews occurred on different dates. The following interview questions 
were asked: 

1. What are the main needs while the facilitator uses the EPSS guidebook? 
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2. What are the problems the facilitator encounters when following the EPSS guidebook? 

3. What does the facilitator expect from the systemic change EPSS to solve these problems? 

4. When and where does the facilitator expect to use the systemic change EPSS? 

5. Are any materials other than the guidebook needed as resources for the change process? 

6. What should the content of the systemic change guidebook be? 

7. What other ideas does the facilitator have to add to the systemic change EPSS? 

After the interviews, the interview notes were transcribed onto 3x5 index cards for each of the interview 
questions. These cards were analyzed and compared, and gradually categories emerged. Once the development team 
members agreed upon all the categories for each question, they carefully defined the categories. Next, the specific 
responses from the interviews were coded by category. At the end of this process several design specifications were 
established for the development of the systemic change EPSS. 

The most important of these specifications is that the systemic change EPSS should make the facilitator 
aware that certain issues are important during the events of the systemic change effort. The next important 
specification is to give guidance on how to deal with those issues if the facilitator does not know how to deal with 
them. It was indicated that facilitators would use the systemic change EPSS before their meetings with the 
participants, rather than during the meetings. 

Specifications related to the user interface, content, and tools are described below: 

User interface related: The systemic change EPSS should be designed with the novice computer and 
Internet user in mind. The user interface should be simple. All pages should include functions to search for the 
information and to print the information. Navigation in the system should be easy and consistent, and transitions 
between the components should be fast. The interface should allow tracking previously accessed screens. The 
interface should inform the user about where the user is in the system. 

Content related: The systemic change EPSS should provide a big picture of the systemic change effort, step- 
by-step advice for each of the 26 discrete events and the continuous events, information about systemic change, 
answers to the Erequently Asked Questions (EAQ), a glossary of systemic change, information about systemic 
change case studies, and interaction with other facilitators. 

Tools related: It was indicated that miscellaneous software tools for collecting qualitative data, for 
highlighting EPSS content and taking notes, and for making casual-loop diagrams are needed by systemic change 
facilitators. 



Components within the Systemic Change EPSS 

There are six main design components in the system. They are consulting, e-guidebook, resources, 
community, glossary and frequently asked questions (EAQ). The consulting component of the system is structured 
as phase and event based. It provides a bird’s eye view of the event to the facilitator. By implementing the consulting 
element, continuous events in the process are embedded in the discrete events, and the facilitator’s consideration of 
continuous events is assured. As shown in Eigure 1, the consulting element provides a direct linkage to resources, 
community and e-guidebook components of the system. 

The e-guidebook component of the system is also structured as phase and event based. The facilitator 
accesses the content of this section either through the navigation system on the interface or through the link on the 
consulting page. The resources component of the system is structured according to the immediate needs of the 
facilitator during the analysis phase of the system. Due to low availability of the resources for each event, the 
resources section is grouped according to the phases in the change process. Community is another component that 
can be accessed immediately from the consulting page. In the community component of the system are directory and 
threaded discussion groups. Erom the consulting component, there are direct links to reach the community 
component as shown in Eigure 1 . 
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Figure 1 . Interconnections between the Components 



User Interface Related Components 

The interface of the system is based on the frame feature of HTML. Using the frame feature enables 
designers to make changes on the interface independently from the content. Most of the web interface designers are 
against using frames on the interface. However, because of some technical reasons we have used frames in the 
system. The interface consists of two frames, a navigation frame on the left and a content frame on the right (Figure 
2 ). 

1. Navigation Frame: The navigation frame is on the left side of the interface. It is designed based on JavaScript from 
the http://www.treemenu.com site. Tree menu navigation has several advantages. Since it is independent menu 
software and it is placed in a frame, it is very easy to add new elements to the site. One of the fascinating features of 
the site is updating the navigation frame within a few seconds without changing any pages. The tracking method of 
the menu, which is opening the folders according to their level in the entire navigation system, provides the user a 
powerful navigation system. Opening the sections folder under a folder reduces the cognitive load of the users and 
provides a clear sense of location for them. 

2. Content frame: The content frame is placed on the right side of the interface. This frame contains the formal 
decisions of our functional decisions. At the top of the page, the main title indicates the name of the site. Under the 
main title, functional buttons are placed. The search field is an indispensable feature for this site, since there are a 
huge number of pages in the site. Even though navigation is so clear and easy, sometimes finding specific 
information through the search field is easier. The “Save and Logout” button helps remind the users where they left 
the site when they come back. Each page has a print button, which generates a printer-friendly version of the page on 
the screen. Under the function buttons, the section title appears. Basically, the section title tells the users on which 
event they are. The content field contains the text from the guidebook. The text is formatted according to the format 
of the original text. 
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At the bottom of the page is the footer bar. The footer bar holds the contact information, sitemap and 
privacy links. Each link opens in a pre -sized window in order for the users to see only related information. The 
contact link contains sufficient information to contact the site administrators via phone, e-mail or mail. The site map 
presents a big picture of the site, and a privacy note explains the copyright issues related to the site. Below the footer 
bar, standard information about the page is provided, such as the page URL and update date, and since the site is 
sponsored by Indiana University, credit is given to the university. 




Figure 2. Systemic Change EPSS User Interface 
Content and Tools Related Components 

The Internet-based Systemic Change EPSS has 6 main components: 

1. Providing guidance to the facilitators: This component constitutes the nutshell or kernel of the system (see Figure 
3). This component is event based. The facilitator sees a big picture of the event and also the important issues about 
the event. It gives a kind of bird’ s eye view to the facilitator of what is going to take place in this event, what issues 
are important, and the place of the event in the whole systemic change process. The facilitator is able to access to the 
other parts of the system from this component. 

2. Electronic version of the systemic change guidebook: The guidebook is designed hierarchically in phases, events 
and activities. The book has 6 phases, each of which contains a number of events, and each event has activities. This 
component provides information about the events, such as things that should be done in order to accomplish the 
objectives of the event. In this section, the electronic version of the guidebook is provided as the main source of 
information. Objectives, procedures and activities are accessible through this component of the system. 
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Figure 3. The View from Systemic Change EPSS for Consulting Page for Event 2 
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3. Resources: This component of the site holds the resources that are needed to enhance facilitators’ skills needed in 
the systemic change effort. These resources are categorized as readings, videos, case studies, sample documents and 
software tools. The site coordinator maintains these resources and updates them as required. Facilitators are not 
currently able to add to the resources. 

4. Community: This component’s objective is to bring people involved in systemic change all over the country 
together in one virtual place. The community component enables people to share their experiences, questions, 
concerns, resources, and documents related to their systemic change efforts. It has a directory of people who are 
facilitating different systemic change projects in order to introduce the members to each other. The component also 
enables school districts to find a facilitator who has experience with similar contexts. One powerful feature of this 
component is the threaded discussion groups that keep the questions and their answers together. The discussion 
software can be customized according to the topics’ needs. 

5. Glossary: This component provides a brief summary of the process of systemic change. It is for novice users who 
want to learn more about the systemic change process and related terms. The component has two sections: terms and 
the article about systemic change by Jenlink et al. (1996). The terms section provides a glossary of concepts in the 
systemic change process. 

6. Frequently asked questions (FAQs): This component has questions that are asked over and over again, and their 
answers are provided by experts in the field. The content for this component is compiled from the community’s 
discussion section and from questions asked directly through e-mail messages to the site manager. 

Benefits of the System 

The Internet-based EPSS for systemic change facilitators provides more knowledge to its users than a 
regular website. With a very simplistic view, it may seem to be similar to any regular web site. However, it differs from 
static websites in several aspects. The important feature of the site is that it provides guidelines for each event in the 
website. Under the consultation link, event-specific guideline pages are provided. These pages contain all 
information related to the events that the facilitators need. A second aspect is that the EPSS has community building 
tools, such as threaded discussion groups and a members’ directory. These tools add value to the EPSS and give 
power to the users to reach each other easily and to share their knowledge. Also, by gathering the knowledge from 
facilitators who are using the website, it provides an invaluable knowledge base for the community. 

Since the system is using online technology and disseminates its knowledge via the Internet, maintenance 
and update costs are relatively low in comparison to printed or optic media. The technology behind the site allows 
the administrators to maintain and update the site without much software knowledge and programming skills. The 
interface design of the site allows the administrators to change it easily. Even drastic changes can be implemented 
within minutes as long as the two frames are preserved. The infrastructure of the site uses DHTML and basic HTML, 
thus it does not require downloading high-end browser plug-ins such as Elash player or Shockwave. 

Usability Tests and Results 

Usability testing involves observing people attempting to achieve specific goals within a specific context. 
Design problems that hinder the ability of facilitators to use the Systemic Change EPSS may also hinder the adoption 
of this system. Therefore, to figure out such design problems and to improve the usability of the system, several 
usability tests were conducted on one facilitator and two non-facilitator users . 

Setting 

The usability tests were conducted with one tester, one computer and one participant. The tester was one of 
the designers of the system. The screen resolution was 1024x768 and the browser was Microsoft Internet Explorer 5.5. 
The tests were conducted in a private place, where the users could spend as much time as they wanted on the tasks. 
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Procedure 

Before the test was conducted, the goals of the test and background information were explained to the user. 
Also the users were advised that they could terminate the test at any time. Then pre -prepared tasks were given to the 
user and the user performed the tasks in the given order. While the test was conducted, the tester only observed and 
took notes. After the test, the tester conducted a follow-up interview with the user. 

Tasks 



During the test, a pre -prepared task list was given to the user. The list included a variety of common tasks 
that a facilitator would do while using the system. Some of the tasks in the list were: starting with the guidance page 
and continuing to an event or activity, finding a specific activity, finding the contact information of the designer and 
posting a message to the threaded discussion group. 

Results 



Most of the pre -prepared tasks were completed by the users in a reasonable time. In the follow-up 
interviews, the users were asked about the quality of the interface and to what degree they felt comfortable using the 
system. Most of the users stated that the interface provided a clear sense of navigation. They did not feel ambiguity 
while navigating through. It was stated that all the tools needed were on the interface and one click away from them. 
All users stated that they would prefer a different font type and font size for the content part. 

Conclusion 

Improving the quality of education is always an objective for any educational system. The guidebook 
provides a guide for facilitators to transform their educational system in a school district. In order to provide more 
convenient and up-to-date knowledge for the facilitators and to establish a community among them, an EPSS was 
designed by a team in the Instructional Systems Technology department at Indiana University. The distinctive 
features of this Systemic Change EPSS are to provide more guidance, to provide an electronic version of the 
guidebook, to allow the users to share documents and to have a directory of facilitators. 

The system consists of six main sections: guidance, electronic version of the guidebook, resources, 
community, glossary and frequently asked questions. The guidance section provides a general picture of the events, 
event-specific tips and guidance for the users about utilization of the event. The electronic version of the guidebook 
provides the book contents to the users. Resources are the documents and tools that are useful to facilitators. In the 
community section, a directory of members and threaded discussion groups are provided. The glossary and 
frequently asked questions sections provide auxiliary information about the transformation process. 

The system adds several benefits to the guidebook. Since it is Internet-based, the facilitators can reach it 
anywhere in the world, and updating the guidebook is easy at anytime and at low cost. Systemic change of 
educational systems requires experience and knowledge accumulation. The Systemic Change EPSS provides this 
knowledge and experience accumulation and it disseminates the knowledge to people who need it. 
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Conceptual Tool 
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Abstract 

We introduce a qualitative method that enables researchers to understand the nature of progressive discourse by 
focusing on how argumentation manifests itself in the collaborative activity of network-enabled communities. We 
adopt a socio-constructivist theoretical standpoint, drawing from socio-cognitive approaches to learning and 
communication, and integrating contributions from argumentation theory, namely informal logic. The method 
consists of capturing a number of logical and logico-grammatical operations in order to identify (1) attitudes that 
might have an impact in collaborative behavior, (2) logical procedures that reveal the nature of the inquiry, (3) 
instances of arguments to understand how thoughts are structured, and (4) relationships between instances of 
arguments across messages. We illustrate the application of this method by presenting a study of the online 
collaborative process of a network-enabled community of practice of nurses. Results are indicative of problem- 
solving, conceptual change, learning, and knowledge building. 

Introduction 

This paper presents a method developed to analyze online conference transcripts, and more specifically 
argumentation processes. While the value of using electronic conferencing is increasingly recognized, studies that 
employ conference transcript analysis methods do not yet form a consistent ensemble; goals are often very different 
as are theoretical perspectives and ways of defining collaboration. The result is a heterogeneous corpus of 
inconsistent scientific research. In our understanding, socio-cognitive research is most promising when it comes to 
communication, learning and knowledge building (Bmer, 1994; Scardamalia and Bereiter, 1994; Campos, 2000). At 
best, electronic conferencing systems enable learners to further explore knowledge, to restructure it, to deepen 
understanding, and to build together (Bereiter, 2002; Scardamalia, 2002). We present in this paper a method that 
enables researchers to study discourse in electronic conferencing systems with a view to describe how the 
argumentation process may lead to manifestations of collaborative problem-solving, conceptual change, and 
knowledge building. To demonstrate the method, we present quantitative and qualitative data collected in a network- 
enabled community of practice of nurses (heart care) that uses Knowledge Fomm® software to exchange information 
and ideas, and to build knowledge. 

Theory and Method 

The method draws from socio-cognitive approaches to learning (Bereiter et Scardamalia, 1987) and 
communication (Grize, 1997; de Kerckhoeve, 1997; Breton, 1996), enhanced by new technologies (Scardamalia et 
Bereiter, 1994; in press), and integrating contributions from argumentation theory (Van Eemeren, Grootendorst, and 
Henkemans, 1996), namely informal logic (Toulmin, 1958). 

Communication, Sociocognitive Sciences and Learning 

Although computer technologies developed steadily from the 1950’s, communication processes have only 
been supported by computers since the early 1980’s, then subsequently became more widely spread in the 1990’s 
(Harasim et al, 1995). With the Internet, synchronous (real-time) communication systems like chats and video- 
conferencing, and asynchronous (not in real time) communication systems such as e-mail, newsgroups, and 
conferencing systems developed. Both kinds of Internet-based communication systems have led to the emergence of 
network -based (virtual communities) and network-enabled communities (e.g. professional communities or classroom- 
based learning communities). 
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There is an obvious need for better understanding of asynchronous networked communication (Campos, 
1998, 1999, 2000; Campos, Laferriere, and Harasim, 2001; Campos and Laferriere, 2002) in community contexts in order 
to enhance learning, problem-solving, inquiry, and reflective practice (Lamon, Scardamalia and Laferriere, in press; 
Breuleux, 2001; Breuleux, and Laferriere, 2000; Bracewell, Breuleux and Le Maistre, 2000; Laferriere et al, 1999; Gibbs 
1994), and to evaluate “what effective learning strategies and modes of learning make the most use of the capabilities 
of the new technologies’’ (Haughey, 2002, p.l8). 

Cognitive scientists and psychologists have studied problem-solving processes extensively, but mostly in 
well-defined domains such as mathematics. Usually, the procedures to solve these kinds of problems follow clear 
mental routes (or strategies) that enable artificial intelligence (AI) researchers to model them in software interfaces. 
Our knowledge is far less developed in ill -defined domains. The approach in this paper is to investigate how 
hypothetical procedures such as those of argumentation, a critical process for communication (Breton, 1996) 
occurring in a network-enabled community, can be scaffolded toward online collaborative problem-solving and 
knowledge building in ways consistent with findings in related studies (Scardamalia, 2002; Hewitt, 2001; Bereiter and 
Scardamalia, 1996). 

A basic assumption is that argumentation is a process that emerges naturally from human cognitive 
structure through language and discourse. Our working hypothesis is that, in the context of electronic conferencing, 
this process can be scaffolded through appropriate facilitation strategies and electronic conferencing tools. Here, an 
argument is defined as a conditional structure (If-Then) in which a number of premises lead to a conclusion, and 
whose content should be understood in the context of its practical use (Toulmin, 1958). To be built in the context of 
the practice through electronic conferencing, an argument needs to be co-constructed (Grize, 1997). Arguments, 
therefore, request responses and active engagement throughout the messages that are posted and that form the 
online discourse. Another assumption of this study is that coming together for the purpose of co-constructing 
arguments for the sake of problem-solving or inquiry is likely to lead to significant collaboration and, hence, to 
conceptual change and knowledge building (Campos, Laferriere and Harasim, 2001; Bereiter, 2002). 

We understand network-enabled collaboration as a process of engagement and active participation in 
which two or more people, through the use of adequate software, build on the thoughts and ideas of one another. 
Network-enabled collaboration can be of different levels: from a vague sense of being together to strong procedures 
of co-work (Campos, Laferriere, and Harasim, 2001). Knowledge building represents a continuous process of 
production and improvement of ideas that are valuable for a community; it is a process in which the sum of individual 
contributions will be less fundamental than what the community would able to accomplish by engaging in knowledge 
production, creation, improvement and innovation (Scardamalia and Bereiter, in press). 

Transcript Analysis Methods 

Most existing methods that aim to study online discourse rely on quantitative measures (Anova, Manova, 
percentages, etc.) of qualitative categories. Studies have a variety of focuses, including critical thinking processes 
(Anderson and Garrison, 1995), analysis of patterns of participation (Howell-Richardson and Mellar, 1996; Bullen, 
1998), content analysis (Mowrer, 1996; Hara, Bonk and Angeli, 2000), teaching styles (Ahern, Peck and Laycock, 
1992), social construction of knowledge (Gunawardena, Lowe and Anderson, 1997; Kanuka and Anderson, 1998), 
notions related to Vygotsky’s zone of proximal development (Fahy et al, 2000), and even argumentation (Marttunnen, 
1997). In addition, the unit of analysis varies from the phrase to a whole text (Rourke et al, 2001). 

Having worked with transcript analysis for a number of years, we searched for reliable tools to study not 
only the content of the conferences but also to understand the argumentation procedures underlying the online 
discourse. The development of this qualitative method is grounded in socio -constructivist perspectives, and studies 
of logically structured discourse (Toulmin, 1958), meaning making (Piaget, 1991), and conceptual change (Piaget, 
1976). 

Analyzing Transcripts of Networked Collaborative Argumentation 

Our method consists of capturing a number of logical and logico-grammatical operations commonly applied 
in written discourse. Depending on the level of their use, such operations may indicate major occurrences in the 
online discourse such as (1) attitudes that may have an impact in collaborative behavior (Campos, 2002a), and (2) 
logical procedures that may reveal the nature of the inquiry (Campos, 1998; 2000; 2002a). In addition, the method 
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captures the (3) nature of arguments in order to understand how thoughts are structured (Campos, 2000). 
Furthermore, the method establishes (4) relationships between arguments across messages. In order to explore the 
socio -cognitive nature of collaborative conversation, the method adopts the phrase as the coding unit. However, to 
capture the knowledge-building collaborative online discourse, the relationships between meanings, as explicitly 
expressed within the coding units of the messages, are the units of analysis. 

Analysis requires a four-step process. The first step consists of identifying when the phrase is of a 
declarative nature (an affirmative, a negative, a conditional or a disjunctive declaration), of an imperative nature (here 
the imperative form of the verb is used), or of an interrogative nature (a question, which, in logical terms, is often an 
inverted hypothesis). Our earlier studies have shown that most online discourse is mainly declarative and that the 
level of use of imperative could reveal a lesser extent of collaboration and even indicate potential conflict (Campos, 
2002a). On the other hand, the more questions formulated by participants, the more engaging the conversation seems 
(Campos, 2000). 

The second step consists of identifying some logical operations underlying the discourse such as 
affirmations, negations, conditionals, and disjunctions (either-or, and or-or operations). Our studies have shown that 
while affirmations do not usually trigger more engaging conversation, negations and conditionals do so, leading to 
collaborative learning, problem-solving, and knowledge building (Campos, 1998; 2000; 2002a). 

The third step consists of identifying the main components of arguments such as: 

- Claim: affirming something that will need data to sustain the argument; 

- Data: information necessary to support the claim; 

- Warrant: hypotheses put forward that either lead to a claim or that derives from a claim. 

These categories were inspired by those identified by Toulmin (1958). The goal here is to understand how 
people structure their thoughts in order to present ideas and build knowledge. 

The fourth step (4) consists of identifying key meanings (ideas) or themes (Grize and Pierault-Le Boniec, 
1991) in the messages, and the interrelationship of instances of argument between messages. The assumption here is 
that meanings are carried out in online discourse through implications. This technique derives from the meaning 
implication analysis, already applied in other studies (Campos, 1998; 2000; 2002a). Implications of meanings are 
expressed by the formula: “if a meaning C is part of a meaning B which is part of a meaning A, then A implies C in 
terms of meanings’’ (Piaget, 1991). We proceed by building a conceptual map to identify how ideas were improved 
(through meaning implications) and to describe processes of collaborative problem-solving and knowledge building. 

The first three steps can be seen as a micro-analysis that provides necessary information to enable the 
researcher to draw the bigger picture of collaborative problem solving and knowledge building. 

Application of the method 



Context 

The research was carried out in collaboration with the Order of Nurses of Quebec (OIIQ), Canada, who has 
partnerships with a number of Canadian hospitals in the provinces of Quebec, Ontario and New Brunswick, and with 
the Francophone Center for the Informatization of Organizations (CEFRIO) who is instrumental in integrating new 
technologies into a number of professional health organizations that provide heart care, such as health and research 
centres and as hospitals. The Order was specifically interested in promoting ways to engage nurses with an expertise 
in heart care to share knowledge as a means to advance nursing practice (Leprohon, 2001) with the help of new 
technologies (Dubois, 2002). The reason for such a decision is that heart diseases are responsible for 38% of deaths 
in Canada (Statistics Canada, 1996, referred by Health Canada). During a six month period, a pan-Canadian network- 
enabled community of practice comprised of 34 French-speaking nurses engaged in problem-solving and knowledge 
building. Participants discussed ways to better nursing practices using adequate technology (Russell, 2002). The 
software used for communication was Knowledge Forum®. 

Knowledge Forum® is a research-based conferencing system, developed to enable collaborative knowledge 
building (Scardamalia and Bereiter, in press). It has web and client versions. The community of nurses used the client 
version, one that allow users to see a graphical representation of the underlying tree-like computer structure, giving 
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the impression that learners work on a neuronal web. In addition to features common to many conferencing systems 
(like message key-wording, word search), Knowledge Forum® incorporates some unique affordances: 

- Editing -users can edit their messages, and now videos, even after they are posted. Different colours 
provide visual cues for new, read, or edited messages; 

- Annotation - users can annotate messages through the creation of embedded text boxes; 

- Rise-above - users are able to synthesize a number of messages that are related by meaning, by collecting 
them into a separate folder; 

- Scaffolds - users can build and insert tags within the text. Five tags, four of them related to argumentation, 
were negotiated between the researcher and the nurses, in line with participatory design (Silva and Breuleux, 1994; 
Gunderson, Currie, and Campos, 1999), and one meant to e discourse structuring (Campos, 2002b). Those tags were: 

1. Claim: introduction of a contextual situation that expresses concerns or difficulties encountered, for 
instance, in professional practice, affirming something; 

2. Data: introduction of facts, statistics, scientific data, research results or other works that have an 
influence on the practice and would support the claim; 

3. Envisaged solutions: engaging in a process of hypotheses formulation that might respond to the claim 
that was put forward and is consistent with the data; 

4. Questioning: formulation of interrogations; 

5. Opinions: offering judgments concerning claims, data, questioning or envisaged solutions presented. 
Data 

During the six month period, 545 messages and approximately the same number of annotations were 
generated. For the purpose of this study, that is, to understand the nature of online progressive discourse by 
focusing on argumentation, we chose two excerpts from two forums: one in which problems related to heart care 
practices were identified (16 messages corresponding to 1 1.51% of the 139 messages of the forum), and another 
excerpt in which nurses worked together to develop content addressing the problems identified (19 messages 
corresponding to 13.29% of the 143 messages of the forum). In the first excerpt, nurses explored issues and proposed 
strategies for encouraging patients to take responsibility for prevention of heart failure, and for their treatment. In the 
second one, the nurses prepared a teaching instrument to be handed to patients to help them to control symptoms 
and signs of heart failure and therefore to enable auto-surveillance. 

Results 

Descriptive analysis 
Step 1 

In the selected thread of the first forum, declarative phrases accounts for 70%, no imperatives were found, 
questions account for 13%, and ambiguous phrases for 17%. In the selected thread of the second forum, declarative 
phrases account for 32%, imperatives for 13%, questions for 13%, and ambiguous phrases for 42%. The high number 
of ambiguity in both threads relates mostly to the presentation, by the nurses, of scientific references taken from 
other authors and a small number of phrases without verbs. If we exclude ambiguity, in the thread of the first forum 
we have 84%, 0% and 16%, and in the second 55%, 23% and 22%. 

Step 2 

In the thread of the first forum, affirmations account for 71%, conditionals for 9%, negations for 3%, and 
ambiguous phrases for 17%. In the second thread, affirmations account for 42%, conditionals for 16%, negations for 
1.5%, disjunctions for 0.5% and ambiguous phrases for 40%. Ambiguity here has the same sense as above. If we 
exclude it, in the first thread we have 85%, 11% and 4%, and in the second 71%, 26%, 2% and 1%. 

Step 3 

In the thread of the first forum, claims account for 18%, data for 69%, warrants for 12%, and eliminated 
phrases for 1%. In the second thread, claims respond for 4%, data for 50%, warrants for 11%, and eliminated phrases 
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for 35%. Eliminated phrases were those that did not pertained to arguments. If we exclude them, in the first thread we 
have 18%, 70% and 12%, and in the second 7%, 77%, and 16%. 

Inter-coder reliability based on Miles and Huberman procedure (two coders) was achieved with the 
following results: 92.69% for step 1, 92.64% for step 2, and 91.92% for step 3. Coding criteria were established by 
applying strict definitions for the categories and a scale of category prevalence based on operatory logic (Piaget, 
1976b) for use in ambiguous cases. 

Conceptual analysis: Step 4 

Quantitative data presented above reveal that in the thread of the first forum most phrases were declarative 
although a significant amount of questions, revealing intentional inquiry, were found. In line with these data, most 
phrases were affirmations although the numb er of conditionals was not impressive and that of negations 
unimportant. If we look at the way arguments were built, it is not surprising to verify that claims and data to support 
claims were prevalent. However, although the level of hypothesizing was modest (shown by the triad questions- 
conditionals-warrants), it shows, nevertheless, that nurses do not rely on guesses, even informed ones. Their daily 
work follows strict scientific rules, and we observe that in their discourse they searched for reliable information 
(shown by the triad declarations-affirmations/negations-data) to support their points. 

The conceptual map of the first forum thread reveals that discussions were triggered by a message 
presenting the following pair: patients should take charge (claim) and the hypothesis if they have to do that then 
nursing strategies should be adopted (envisaged solution). Around this pair of arguments, five main themes were 
linked either to the claim (Best models in heart re -adaptation. Convincing, and “Rewards”) or to the hypothesis 
(Contracts and The pair counseling/teaching instrument on heart care): 

1. Contracts - in the 7 messages of this thread, most discussions were triggered by statements made on 
phrases coded warrants (hypotheses). The nurses explored the possibility of contracts to promote 
patient responsibility for their own health care: Should it be mandatory? How could a contract engage the 
ill? Is a contract a real solution? Is a contract motivating? Is a contract an instrument of awareness? What 
would be the role of doctors in such a contract, that of controlling or one of engaging? How critical is the 
implication of doctors if such an idea was to be adopted? The responsibility of doctors was further 
explored by a “thread” of annotations built around those messages. 

2. Best models in heart re-adaptation - in the 2 messages of this thread, most discussions were triggered by 
statements made on phrases coded as warrants. Many models of heart re -adaptation were presented as 
well as proposals of integrating many models or adopting one of them. This theme was extensively 
discussed in the discussions surrounding the following themes “Convincing” and “Rewards”. 

3. Convincing - one single message presented this theme as data, through the presentation of a case. A 
“thread” of annotations was built around this message to hypothesize what would be the “x” element to 
convince a person of the need to take care of heart health, how to motivate, and how to understand 
resistance to care. 

4. “Rewards” - a thread of 4 messages and a “universe” of 1 1 annotations was built to discuss cases and 
the need to provide “rewards” that would serve as a motivating element for the ill to take charge of their 
health. Here, ideas were equally triggered by statements coded as warrants as well as data. This theme 
originated discussions around ideas for rewards, the implications of rewards, how to motivate, 
psychosocial factors involved in motivation, in addition to strategies to enable this “technique” and 
models of behavior change that should be adopted. 

5. The pair counseling/teaching instrument on heart care - one annotated message is built around an 
envisaged solution according to which teaching instruments, such as a guide of auto-surveillance, might 
be considered as a kind of counseling. 
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Concerning the thread of the second forum, quantitative data presented above reveal that although most 
phrases were declarative, they were not as prevalent as in the first thread, and that an important number of questions 
and imperatives were present. Imperatives were used primarily for the preparation of the auto-surveillance instrument 
(example: Press your leg; Look at your finger, etc.). It is striking to note that, although most phrases were 
affirmations, the amount of conditional reasoning found was high, followed by a small number of negations and 
disjunctions. The argumentation process, however, was built upon a lesser number of claims and an increased 
number of data and warrants. Here, we found that the level of inquiry (shown by the triad questions-conditionals- 
warrants) was lower in the thread of forum 1, created to explore and to identify problems, than that of the thread of 
forum 2, in which nurses agreed to build, in collaboration, teaching instruments for patients. Thus, the data reveal 
that when working collaboratively on concrete objects related to their practices (thread in fomm 2), nurses were more 
able to reflect, to inquire, and to hypothesize. It should be noted that the number of annotations (whose use 
demonstrate that they could be posted as full messages in the forum, and not published as simple annotations) is 
much higher (34) than the number (19) of threaded messages. In addition, annotations were mostly triggered by 
message phrases coded data. Furthermore, one quarter of the annotations were made of hypotheses that instigated 
further hypotheses formulation in the messages. Finally, the other annotations were equally divided between claims 
and data. 

The thread of the second forum should be seen as a continuation of the first. This continuum can be seen 
as a problem solving process; nurses moved from a phase of problem identification to a phase in which they would 
work on a deliverable to provide a practical solution to the needs of the ill: an auto-surveillance instrument kit. In this 
thread, discussions were triggered by a message presenting the hypothesis if an instrument should be built to 
enable patients’ auto-surveillance then what signs and symptoms should be addressed and which actions should 
be taken when they identify them? (envisaged solution). Linked to this message, we find three themes (The pair 
priority information and scale. Building the instrument. Learning auto-surveillance), and a Rise-above (on Symptoms): 

1. The pair “priority information and scale” - in this message, surrounded by many annotations, nurses 
listed priority information for the auto-surveillance instrument and hypothesized which, and how, scales 
would enable patients to evaluate their own health. Claims, data, and warrants were equally balanced in 
the annotations. 

2. Building the instrument - in the 5 messages of this thread, careful attempt to build the instrument led to a 
discussion about the adequacy to create a calendar that will serve as patient organizer of the auto- 
surveillance process. Most messages are hypotheses {warrants), followed by claims and data. 

3. Learning auto-surveillance - these 4 messages, surrounded by 23 annotations, discussed extensively 
different aspects of the instrument such as different kinds of scales, their advantages and disadvantages, 
useful suggestions, how to address the vulgarization of scientific information, symptoms such as fatigue 
and increased weight, among others. The discussion was mainly constructed around (fata presentation 
but claims and warrants were equally important in messages as well as in annotations. 

4. Rise-above - a message-synthesis including 8 new messages and a number of annotations served the 
purpose of collecting messages whose main themes were signs and symptoms, their definitions, types, 
occurrences, etc. A few mes sages, included in the three themes presented above (The pair priority and 
scale. Building and instmment, and Learning auto-surveillance) and discussing signs and symptoms, were 
also included. 

The conceptual map of the forum 2 thread is evidence of the power of the strategy implemented. A network- 
enabled community of practice of nurses facilitated by a nurse with an expertise in heart care in regular contact with a 
facilitator from the action-research team, and supported by software (Knowledge Forum®), engaged participants in 
knowledge building. 
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Conclusion 



When people use language on a daily basis, they spontaneously structure their thinking. When using 
Knowledge Forum®, the situation is analogous, but with the difference that the discourse is written. People’s 
thoughts and ideas are naturally structured by subjacent logical operations that are expressed by grammar and by the 
way we build arguments. Therefore, claims, data, and warrants are naturally interwoven in the process of 
argumentation. However, it is worth noticing that network-enabled exchanges only happen when people demonstrate 
engagement by reading and writing notes. This is, essentially, a process of inference making about what others 
mean in the messages. Inferences are If-Then operations. That is, “if’ this A is written “then” my reaction to this is B, 
and intentionally I will respond to this by writing C. The data analyzed through our method show that this 
inferencing process happened at four levels: 

a) Application - intentional triggering of collaborative action by the vulgar mechanism of replying to a 
message (building on) but also by the annotation tool, the rise above, and the scaffolding tool; 

b) Unintentional use of logical and grammar operations - comparison of the first and second forum threads 
shows how the nature of a task alter participants’ unknowing use of logical and grammar operations 
subjacent to discourse. For example, in the second thread few claims were made because what was in 
question was the building of a kit that requested mo stly data presentation and reflexive thinking; 

c) Intentional use of argumentation tags - although we do not present here quantitative data related to the 
use of scaffolding tools, the way participants applied scaffolds was in most instances consistent with our 
own coding. Concerning the two scaffolds that were not directly related to our coding scheme, 
“questions” were considered instances of warrants while “opinions”, depending on the context, 
functioned as claims, data or warrants; 

d) Intentional idea linking and improving - most links among messages ( ideas or themes ) were originated in 
phrases coded as warrants, which were either questions or formulated hypotheses, although the 
importance of inferencing upon claims and data put forward should not be minimized. This aspect is 
indicative of progressive communication and discourse leading to learning, conceptual change, and 
knowledge building. The progressive discourse of participants is rich in showing how ideas evolved and 
how nurses enhanced their own practices by applying - and sharing with others - both practical and 
scientific knowledge. 

This method enables researchers to study progressive discourse through argumentation with a view to 
understanding how collaborative processes of networked-enabled communities engaged in problem-solving lead to 
conceptual change, learning, and knowledge building. 
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Abstract 

The staff of the P3T3 project have a primary mission to train and mentor faculty members in the School of 
Education as they assimilate technology in instruction. This study investigates the evolution of select faculty 
members; how did they 1 ) integrate technology, 2) perceive technology, and 3) implement technology effectively? 
In addition, existing problems in faculty training and inefficient processes in technology integration are 
investigated. The authors provide suggestions for early prevention of similar future problems. 

Introduction 

During the past two and a half years Purdue’s Program to Prepare Tomorrow’s Teachers to Use Technology 
(P^T^) has been preparing teacher education faculty to teach pre -service teachers in technology-rich environments by 
modeling approaches that future teachers should use themselves. In order to achieve this goal, P^T^ incorporated a 
comprehensive faculty development and mentoring program, which is as individual as the faculty members 
themselves. P^T^ combined workshops, training sessions, and one-on-one mentoring to allow each faculty member 
the opportunity to use their own learning style to expand their use of technology in the classroom. Workshops 
consisted of a two-day "start-up" session and were held at the beginning of each semester and during summer 
breaks. Participating faculty members engaged in problem-based learning activities involving technology integration 
and previewed future offerings of technology skills development workshops. They also had opportunities to view 
models of technology integration in education. At the conclusion of the "start-up” sessions, faculty members 
developed and shared personal technology integration plans. Based upon the technology integration plan, a 
graduate student - skilled in the appropriate software/hardware to meet the goals of the faculty member - was 
assigned to provide optimal mentoring throughout the integration of technology into teaching during the 
subsequent academic year. Technology skills development workshops have been held (and are still being offered.) 
which include workshops on WebCT, PowerPoint, FrontPage, and Dreamweaver, and “how to’’ sessions focusing on 
the use of digital video, digital photography, HTML programming language, and the School of Education’s ePortfolio 
system and the Purdue career account system (supporting centrally managed storage for files and web pages). 

Through these workshops and personal mentor training provided by the P^T^ graduate assistants, some 
faculty made great strides in the development of technology skills. Others attended the workshops but encountered 
major struggles and challenges as they attempted to incorporate technology into their courses. A few faculty 
members chose to continue their curriculum in the same manner as it always was - without the addition of new 
technology. This study investigates the evolution of select faculty members; how did they 1) integrate technology, 

2) perceive technology, and 3) implement technology effectively? In addition, existing problems in faculty training 
and inefficient processes in technology integration are examined. 

Theoretical Framework 

Considering the rapid pace of development in educational technologies, coupled with innovations in 
teaching and learning, utilizing technology is high on the list of priorities of educational managers, administrators and 
operatives. Sandholtz et al. (1997) noted the use of technology is recognized as a valuable tool - making technology 
more common while developing it to enhance teaching and learning. However, Brand (1998) found that despite 
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increased access to computers and related technology, educators are experiencing difficulty in combining technology 
into classroom teaching practice. Training and mentoring provide two major incentives in aiding faculty to 
successfully integrate technology in teaching (Dusick, 1998; Dusick & Yildirim, 2000). 

Methodology 

This research relied on case studies conducted by two of the authors. Both are proficient technology users 
and worked as graduate assistants in the P^T^ program. They served as mentors for the faculty members and became 
a part of the project at its inception in the summer of 2000. One continues to work with the faculty members through 
the third year in the grant. 

Through an interview process, responses from three faculty members and a teaching assistant were 
videotaped or tape-recorded. Two additional faculty members included in the data were not interviewed, but 
information about their technology use was obtained from interviews with their personal P^T^ mentor. Completed 
faculty projects were also used as a source of information for the subjects - websites and edited video projects 
provide evidence of faculty involvement with technology. Selected subjects included individuals who successfully 
learned and integrated technology as well as individuals who chose not to make changes. The faculty members have 
been teaching from 10 to 37 years in K-12 and higher education. A variety of developmental stages with computer 
skills were targeted One faculty member was proficient with technology, but the others were at different levels of 
expertise. For the purposes of this study, they were categorized into two groups: website designers (using software 
and their own abilities to program the pages), and innovators who attempted to use video in their curriculum or 
course presentations. The interviews were conducted to explore faculty member attitudes toward technology 
integration, the strategies they used to learn the technology and integrate it into their teaching, and to determine the 
challenges they faced as they incorporated the new skills. 

Website Integration 

Three faculty members were interviewed about their website design experiences. Two had no prior 
knowledge of website development but the third supervised a graduate student in the development of her site and 
maintains her own website. 

Successes 

Each member of the faculty who was interviewed met success in their ability to learn the process of website 
programming and management. Some were able to use the tools and make their pages more engaging, but all of the 
sites met or exceeded the faculty member’s expectations. The professors in this study each attended a faculty 
workshop and began their individual website projects because they felt there was an excellent opportunity to 
enhance their skills with a personal mentor. A professor in the School of Education for more than 10 years. Dr. E. 
enlisted the help of her P^T^ mentor she created her own professional web site as well as a second one dedicated to 
her research interests. Professor E. stated, “There are several people I work with that use web pages. . . It’s very 
convenient for me to get information. Web pages look like the information source. The assistance I get from 
P’t\ . .definitely impacted me to have my own web page. I don’t have the technical skills to do it, but after I went to 
the Dreamweaver workshop, I know what I can do.’’ Dr. A. was an administrator in the K-12 school setting for more 
than 30 years. He has been a Professor in the School of Education for four years. Like Dr. E., he created his web 
pages with the help of his P^T^ mentor after attending a faculty workshop. He stated, “I use the web a lot in my work 
area. . .this university is a highly -technology involved university. Both of these are motivations for me to create a 
website for myself.’’ Dr. E. has more than 5 years of experience as a professor in the School of Education. She 
recognized the value of the web as a communication tool for her students, her associates at other universities, and 
the outside world even before the P^T^ project began. She employed a graduate assistant to create her own website 
using ErontPage on a PC, but was often frustrated when she tried to edit the pages using a Macintosh computer 
because the version of ErontPage available on the Mac was not updated at the same pace as PC software. After the 
offer of technical assistance from P3T3, she was excited about converting her designs from the PC system to 
Dreamweaver on the Macintosh platform. With positive reinforcement from her P^T^ mentor and technical help when 
she needed it, she found it possible to transferred the pages to the new format rather than recreating the pages in a 
new website, and has used her own artistic skills to reflect her personality throughout the site. 
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Challenges 

Because of their unique skills, software, systems and desires for programming, some challenges were more 
easily solved than others. Comments from faculty members throughout the department - even those not included in 
this interview are frustrated with the lack of time available to learn technology. Although the workshops are offered, 
they must make arrangements to dedicate their time to attending. Dr. F. stated, “Faculty’s time is very tight.” Dr. E. 
agreed and also stated that she is a Mac user; however some web design software, like FrontPage, is updated only 
on the PC platform. Her need for P^T^ assistance occurred when she found a need to update her current site. She 
approached a P^T^ assistant and even though together they had a great deal of skill and working knowledge of 
technology, they experienced tremendous difficulty making edits, even when working on the platform that was 
originally used to create the pages. As previously mentioned, a grad student had originally programmed the site, and 
he was no longer available to ask about specific programming. This meant editing was neither efficient nor easy. 
Unsuccessful attempts were made to edit the pages in a manner the faculty member could understand and duplicate; 
there were simply too many variables. Nothing was consistent. The P^T^ mentor recognized the faculty member’s 
confusion and frustration as they tried to work on the site together. The mentor chose to work on the site alone, read 
the HTML coding, identify the problems associated with the programming, and save new documents that were easier 
for the faculty member to edit. When the mentor and Dr. E. met again, the new documents were successfully updated. 
The mentor’s decision to work on the project alone reduced Dr. E’s. frustration and prevented possible faculty 
burnout, which would have eliminated the desire to make future changes. If Dr. E. had faced the problems alone, they 
may have been overwhelming. By finding the problems and creating documents that could be more easily edited, the 
mentor helped the faculty member gain experience and confidence. Currently she needs little more than occasional 
help for her updates. 

Dr. A. was already familiar with specific web-editing software but he was faced with a different software 
when he began creating his site because of support limitations. He stated, “The changing of the software forced me 
to learn the new technology. However, it’ll be a long process for me to learn and update the new software.” 

Impact of P^T^ 

In each of the case studies, P3T3 had a direct impact on the success of the faculty project. Their testimonials 
indicate their satisfaction in working with their mentors, and reaching their personal goals in website integration. 
Professor E. stated, “Without the assistance of P3T3, 1 don’t know if I would have created [a site] or not. I got hands- 
on experience, and specific handouts in the workshop; I get back and use it. After the workshop, I got individualized 
help from P3T3. P3T3 provided me encouragement, support, a sense of ‘can-do’ confidence.” Professor A. stated, 
“The P3T3 assistant has been indispensable for me to put that [site] on. [She] helped me a lot. It would not be on so 
quickly, in such a quality without [her] help. I really appreciate [the] help.” In contrast. Professor E. stated, “I went to 
a couple of ErontPage workshops, but they didn’t get to the level and skills I needed. They just start teaching how to 
create a new page, how to insert graphics.” Because she was already proficient in these simplistic tasks, she found 
the P3T3 one-on-one mentoring provided the required, more advanced help. 

As P3T3 mentors work with faculty, the goal remains to help the faculty member determine the correct 
software before their projects begin. When projects are already underway, mentors must watch for signs indicating 
when faculty members become frustrated and search for ways to minimize the frustration while providing alternative 
methods and tools for motivations so that the faculty members will to continue making personal technical 
advancements. Eaculty who already have some expertise but desire to learn skills beyond those taught in beginning 
workshops, have needs very different from the beginner, but those needs must also be met if the project is to be 
successful. 



Video Editing 

During the P3T3 beginning workshops, iMovie on a Macintosh was introduced as a quick and easy tool to 
add video into the curriculum. Several faculty members showed interest in attempting to make a video clip from a 
digital camera that could be used in the classroom or archived as part of a portfolio. Pour faculty members 
subsequently purchased digital video cameras with a stipend provided from the P^T^ project. 14 cameras are available 
in the department, so anyone with the desire to take and edit a video has the opportunity to do so. A variety of 
results occurred from the purchase of these cameras and the faculty recording, editing, and incorporation of digital 
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videos. Four faculty members were targeted for this case study. One faculty member and one teaching assistant were 
interviewed, as well as two assistants working as a faculty mentors. 

Successes 

Mrs. L., a teaching assistant taped an interview with a Master Teacher in a K-12 classroom setting as a 
method to introduce the teacher to her students. By taping the interview, the students could see the teacher in her 
own setting, surrounded by her “teaching world.” Mrs. L. wanted to edit the tape prior to introducing the teacher to 
her students but didn’t realize the commitment of time that was required to accomplish her goal. She was immediately 
provided with personnel resources and training to help her complete the work within her timeline. The result was a 
teaching assistant with a new skill, and a polished final product that succeeded in meeting her objectives. Dr. E. first 
became involved with video through a 3-year project she headed in the Educational Technology Department - a few 
years before the P^T^ project began. Although she had extensive experience with video, she took a more personal 
interest in adding it to her curriculum when she saw the ease of digital video editing as presented in the P^T^ 
workshops. As a direct result, she purchased a digital camera and began working with it herself without the aid of her 
mentor. Soon she was editing the videos and she currently has plans to incorporate video in her classroom in the 
future. 

Challenges 

Even though Mrs. L. was a new teaching assistant in the School of Education, she set her project into 
motion with a set of clearly defined goals. However, she didn’t realize the amount of video-editing time required to 
meet those goals. With her new responsibilities and minimal free time she was overwhelmed with the task of editing 
60 minutes of video footage. Professor G, a 17-year faculty member at Purdue, wanted to use video to document his 
research, but he became apathetic when he found that he could not use his video camera to download still photos to 
his Macintosh due to platform incompatibility. Professor C, a Visiting Professor in the School, was truly interested in 
incorporating video in her student assignments, but she was personally technically challenged. She attended a P^T^ 
faculty workshop session designed to build computer skills through the use of hands-on training but experienced 
difficulty during the sessions. She commented that she did not learn skills when she was guided through each “point 
and click.” Even though she wasn’t comfortable with her own use of the technology, she realized its significance for 
her students so she created a lesson plan which included a video assignment for her undergraduates. While this 
project had great potential, neither she nor her course instructors had the technical skills to support the activity, and 
there were more than 400 students who would need assistance. When it became clear that P^T^ staff would be unable 
to provide support. Dr. C. changed the assignment. She also complained to P^T^ staff that her teaching assistants 
should not be expected to support the technology. She said that they were already too busy teaching course content 
- that “they don’t have the time to teach technology.” 

Impact Of pV 

Each of the four instructors had different reasons to use video - personal research, interviewing guests, 
curriculum enhancement, and to challenge students with technology. Professors G and C do not have plans to 
continue using video at this time. Their lack of research and quick decision-making affected their satisfaction with 
this tool’s integration. Dr. G purchased a camcorder before realizing it did not fit his needs, and Dr. C assigned a 
student project before verifying resources for support. In a similar way, Ms. L. was overwhelmed by the amount of 
work necessary to edit the tape, and didn’t plan enough time to do it alone, but she communicated her needs 
effectively and the P^T^ staff were able to help her meet her goals. Each of these three instructors had limited 
experience with technology. On the other hand, “Dr. E.” is technology proficient and realized she needed to take time 
with a camera and learn to use it prior to adding the component to her classroom, choosing a slower path - and she 
did this without the advise of her P^T^ mentor. 

In the hands of experienced P^T^ assistants, technology appear simple. It was easy for faculty members to 
become motivated to learn a new skill when opportunities arise. However, learning a new skill takes time and 
commitment. Unfortunately; two of the faculty members lost motivation when they learned that their goals had a price 
that was too high (in time and/or the requirement to learn and teach a new skill). Since these experiences, as P^T^ 
introduces new technology, we remember the two faculty members who were successful. We have become more 
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aware of signs that instructors are planning unattainable goals and try to more effectively communicate the time and 
resource commitments necessary to integrate these new technologies so that instructors have a better chance of 
success in developing realistic goals for their future endeavors. 

Outcomes 

This case study shares successes for some faculty, but more importantly, it identifies problems - including 
frustration when using psychomotor “point and click” hands-on workshops, platform-dependent issues, lack of 
awareness of the scope of technology-integration issues, and a need for better pre-planning. When attempting 
faculty training instructors and mentors must analyze their audience/mentees and strive to meet their needs. It easy 
to assume all learners need basic skills . It would be beneficial to let the participants bring questions and specific 
needs or challenges to the workshops. Help them understand what they can achieve at the end of their training, and 
engage them in dreaming of endless possibilities for future integration. When tasks appear to be as simple as a “point 
and click” staff must be capable of explaining the learning curve necessary to become proficient with the 
software and hardware. Most of the workshops are conducted on PC computers, and skills may not translate easily to 
Macs. Although many software programs are seamless today, there are issues when older versions of software are 
used, and these issues should be addressed. Mentors have an obligation to give faculty confidence in themselves by 
helping them determine realistic tasks using reasonable time and effort while using the appropriate equipment to meet 
their goals. Faculty must to be able to practice and use their new skills in their own time on their own computers so 
they are motivated to continue to increase their skills and become independent users of the technology who no 
longer need assistance. With the new skills, they will develop their curriculum to include technology designed for 
teachers of the future, and those of us involved with the P^T^ project will have met our goals. 
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George W.H. Smith III 

Chadron State College 
David M Winograd 
SUNY Postdam 
Abstract 

The concept of structure as part of transactional distance theory is changing. There are differing definitions of 
structure leading to confusion as to what structure is. Structure as an instructional design element in the context 
of a mandated interaction model and moderator training is described. A dynamic structure is designed in the 
moderator training model and a static structure is incorporated in the instructional design for the mandated 
interaction model. The interplay of ideas in the instructional design element of structure in computer mediated 
communication in educational conferencing is compared using two different models. 

Changing technology can dictate differences in how distance education is designed and delivered. Along 
with changing practice comes theoretical change. As Garrison (2000) notes “Theory in distance education must 
evolve to reflect current and emerging innovative practices of designing and delivering education at a distance.” 

At the heart of current theoretical change is the concept of structure as defined by Michael Moore in the 
early 1970’s. In the early 1970s IP centric video conferencing did not exist so theory developed now is specific to 
different kinds of interaction. Dialogue in Moore’s terms is text based. Moore’s transactional distance theory 
pertains to the interplay between dialogue, structure and learner autonomy. It is described as the greater the structure 
and the lower the dialogue in a course the mo re autonomy the learner has to exercise. It defines a pedagogical 
distance related to communication between students and instructors in online courses. In essence Moore’s theory 
states that the more structure the less communication and the greater the transactional distance (Moore, 1989). 

Twenty four or so years later a project was designed by Saba and Shearer using telelessons to provide data. 
Saba and Shearer (1994) attempted to evolve the construct of transactional distance by constructing and 
implementing a system dynamic model where the relationship of designated variables is presented. Saba feels that 
the dynamic and complex nature of distance education requires a system dynamics set of boundaries when analyzing 
relationships of those variables. Saba and Shearer (1994) tested the model empirically with a limited number of 
participants. They analyzed instructional discourse and reported the relationships between transactional distance 
and its two component variables, dialogue and structure. The observations confirm the idea that dialogue and 
structure are found in an inverse proportion to one another. 

In the time between the early 1970’s and now communications technology has evolved to a point where the 
structural constraint of distance is a relatively minor design challenge (Garrison 2000). The minor design challenge of 
designing structure in an asynchronous learning network (ALN ) is discussed further in this paper and references 
text based interaction. 

The kind of structure employed in the design of an online course determines the quality and amount of 
interaction. When a delivery medium for instmction is evaluated for use there are many factors that will influence the 
decision of what to use. The most important factor is how the course is structured by instructional design. 
Instructional design considerations for classroom delivery are different than online courses using Computer mediated 
communication (CMC). When the benefits and disadvantages of delivery of instruction using the Internet are 
examined it is important to understand how student attitudes are affected. Students using web based delivery of 
instruction tend to feel isolated from a learning community. Instructional strategies like using photographs and 
frequent structured interactions can influence whether a student chooses to remain enrolled in a course. Using such 
strategies can make a learner feel like they are part of a learning community. Making a course interactive by designing 
structured writing intensive activities between students and instructor and students and students is important. “The 
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potential for students to increase their knowledge through interaction with other students who are separated by time 
and distance increases as academic discourse is facilitated”. (Smith, 2000 ). 

Dynamic discussions can be used as a structural element of a stand-alone online course or as an online 
component of a face-to-face course. Online courses using structured asynchronous discussions tend to satisfy more 
learners than other kinds of online courses. Shea, P., Frederickson, E., Pickett, A., Pelz, W., & Swan, K. (2000) noted 
that “when students reported high levels of interaction with classmates and high levels of participation in the 
courses, significant correlations were found with high levels of satisfaction and learning. Students who reported the 
lowest levels and quality of interaction with the instructor and other students reported the lowest levels of 
satisfaction and learning. They had the least opportunity for collaboration.” This concept indicates wider research to 
determine if courses are designed with a stmcture facilitating higher levels of quality interaction, will higher levels of 
learning and consequently greater levels of learner satisfaction occur. 

Moreover, as Passmore notes (2002)”leamers need information. However, they also need other instmctional 
features. These include opportunities for reflection and varied practice to promote storage and retrieval of 
information from long-term memory, discussion with peers and the instructor to sharpen their understandings and to 
test ideas, frequent coaching and feedback on performance to correct their paths toward learning outcomes, case- 
based and project-based analysis of concepts, and brief periods of apprenticeship to apply concepts within close to 
real-life situations.” 

A problem in online courses is lack of participation . As Winograd notes “Full and active participation is a 
key to taking full advantage of the collected knowledge and experience of the group. . .”(2002) A moderator can 
motivate a student to participate through thoughtful and careful wording of messages to the non participating 
student. “Learner/instructor interaction provides motivation, feedback and dialogue between the teacher and 
student”. (Mclsaac 1996, p. 407) Interaction between participants requires a very specific structure created by the 
course designer, instructor and/or moderator in order to attain the levels of participation required. Moreover the 
emphasis on structure becomes especially important when there are no strong alpha type personalities who assume a 
leadership role in moderating discussions. There are demonstrated instances where a student with a strong 
personality has taken on the role of moderator in a course where there is unstructured conferencing. In unstructured 
conferencing the structure is dynamic as the moderator interacts with the students. 

In a mandated interaction model the structure is static. The rules for interaction are laid out for the 
participants before the interaction occurs. In this model learners are required to write about a specific topic and are 
instructed how much to write. An important element in an online discussion is peer to peer writing. When students 
are mandated to write for one another they tend to write more thoughtfully than if they are writing just for the 
instructor. It is important to provide a structure for such interactions so there is no question about what is required of 
the student. For example the following specific instruction from an online portion of a course for school 
administrators that mandates use of a threaded discussion. What is it like as a female administrator suggesting policy 
promoting standards reflecting the centrality of student learning to a school board? Post your carefully crafted 250 
word response to this question in the discussion board, unit 5, by the end of the day on Thursday. By the end of the 
day on Monday read all of your colleague’s responses and post a reply of two or three sentences to one of your 
colleagues responses stating why you agree or disagree or what you learned from that response. This kind of 
structure gives the students the ability to represent personal experiences that may be more relevant to the 
understandings of other students than the understanding the instructor is trying to put forth. Moreover the precise 
instruction the instructor provides regarding the discussion can lead to higher levels of perceived learning as 
determined by percentages of course grades (Shea, P., Frederickson, E., Pickett, A., Pelz, W., & Swan, K., 2000). 

Discussion 

Structure as a design element plays an important role in online courses, lack of design centric structure leads 
to lack of interaction. Comparison of cases of design centric structured and unstructured online courses gives a clear 
picture of how a course can fail in its goal to be interactive. Moreover there is confusion about the exact meaning of 
structure. As Garrison (2000) states; “...the exact nature of the interrelationships among structure, dialog and 
autonomy is not clear. There is confusion around whether structure and dialogue are variables, clusters or 
dimension.” Therefore it is important to treat structure as a design element employed to facilitate interaction in online 
courses where computer mediated communication is used. 
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Abstract 

This paper explains the results of an effort to incorporate the constructivist learning theory in a text-based 
traditional distance education program in Turkey. After a series of expert panels, it has been concluded that it is 
quite hard to design a distance learning system completely based on constructivist learning theory in Turkey due 
to many limitations such as large body of students, lack of enough number of facilitators, etc. However, experts 
participated the panels indicated that it might be useful to use some of the principles of constructivist learning 
theory in designing distance learning materials-mainly textbooks. As a result of this, a textbook that includes some 
of the principles of constructivist learning for Anadolu University’s distance learners has been designed and 
distributed to the learners. The survey conducted at the end of the course has revealed that distance learners have 
shown positive attitudes toward the constructivist parts of the textbook. 

Introduction 

It can be claimed that increasing population and changing work life are the main developments that boost 
the demand for distance education. The first one, increasing population, has been one of the main problems of 
Turkish education system for years. Around 70 million -mostly young- people want to have education opportunity 
but mainly due to limited number of institutions and instructors as well as economic problems most are not able to get 
it. For young people, having a higher education diploma seems to be one of the main requirements of getting a decent 
job. So, every year around 1,5 million people take university entrance exams but only 1/5 of them can get in a 
traditional face-to-face program. 

In these circumstances, Anadolu University has been providing the best alternative for the ones who could 
not get in a traditional university program since 1982. Exactly 20 years ago, a school titles as Open Education Eaculty 
established to offer two distance education programs in the fields of economics and business administration to 
mainly people who have a high school diploma but left out of traditional university programs. It started the education 
with 29,479 bachelor’ s degree students. In 1999, the World Bank recognized Anadolu University as the largest 
university on earth with its 504,000 students (MacWiUiams , 2000, p. 2). Eor the 2002-2003 academic year, around 580 
thousand Turkish people had rights to get in a university program according to the university entrance exam. Around 
300 thousands of them registered for traditional university programs. The other 280 thousands were able to get in one 
of the distance education programs of Anadolu University and around 200,000 new students registered to these 
programs. With these students, the current number of the distance learners of Anadolu University reached nearly 
800,000. This number can be considered as an indicator of the effects of increasing population on demand for 
distance education. This number also shows the achievement of Anadolu University in supplying the demand for 
education in Turkey. 

Today, Anadolu University provides education opportunity through its distance programs to the people all 
around Turkey, around Europe, Turkic Republics and in Turkish Republic of Northern Cyprus. Graduates of these 
programs work almost all sectors (private-public, for profit -not for profit, good-service). 

The nature of students who are choosing to get in Anadolu University’s programs is changing, too. Most 
of its students are working in regular jobs and are coming to the programs for improving their skills in their jobs. Of 
course still majority of the students are getting in the programs for having a diploma. 

In the distance programs of Anadolu University, television programs and textbooks are the main media used 
for delivering the instruction. The University produces its television programs in-house and sends to national public 
broadcasting company, Turkish Radio and Television (TRT). TRT airs these programs on one of its channels, which 
is mainly established for educational purposes. The textbooks are also produced in-house. Anadolu University has 
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professional TV program production studios and publishing facilities. In the processes of designing and developing 
these materials media personnel work with content experts and instructional designers. 

Anadolu University always tries to improve its services. Efforts of integrating new media into its system are 
the examples of these tries. For instance, delivering some courses through videoconferencing to the students in 
Kazakhstan and Northern Cyprus, sending video cassettes of television program to the students all over Europe, 
offering an online associate program on information management can be considered as the indicators of the wish to 
improve its services. 

The efforts for improvement are not limited to the media. Anadolu University is also trying to incorporate 
new developments in the theories of learning and teaching into its delivery methods. Instructional designers in the 
course teams are responsible for improving the quality of learning from the course materials (mainly textbooks due to 
wide spread use of this medium). 

This paper is related to one of these efforts about delivery methods. In other words, it explains the results of 
an effort to incorporate the constructivist learning theory in a text -based traditional distance education program. The 
main purpose of this effort, which involves a study, is to examine how the distance education textbooks can be 
designed according to constructivist view and what the distance students’ attitudes are about the textbooks, which 
are designed by using constructivist principles of learning. 

Textbooks and Constructivism 

For years textbooks have been designed to convey the content, or a linear mode of thinking to learner. In 
order to help designers to accomplish this duty, first behaviorist theories have provided detailed prescriptions and 
then cognitive theories offer some additions. 

Recently, however, an alternative view about the learning process has gained attention. Recent studies on 
learning have revealed that learning is a change in meaning constructed from experience, and after bringing his/her 
prior knowledge to the learning experience, the learner co-construct new meaning onto former knowledge through 
interaction with the material, with the other learners and with the instructor. That is why this new approach has been 
called as constructivism, or constructivist learning (Ertmer & Newby, 1993). 

Due to its social nature constructivist learning is usually associated with new computer-based 
communication systems (Vrasidas, 2002). Literature is full of using constmctivism into online learning environments 
(e.g. Jonassen & Rohrer-Murphy, 1999; Lefoe, 1998). However, there is not any study on using only constructivist 
perspective to design and develop a distance education system that involves large group of learners, one-way 
communication, and requires individualized learning. 

On the other hand, current applications have revealed that textbooks will still be used in learning 
environments which is whether constructivist or not and online or face-to-face. Some expert such as Cunningham, 
Duffy and Knuth (2000) indicate that textbooks can be designed to use in constructivist learning environments. 
Although, they point out that future textbooks will be different than today’s textbooks. 

The Study 

In the first phase of the study a list of instructional principles prepared after completing literature review. 
Then for the second face a group of senior instructional designers who work in course materials’ design and 
development teams of Anadolu University brought together for brainstorming in a series of expert panels. The results 
of these panels and literature review revealed that it is quite hard to design a distance learning course completely 
based on constructivist learning principles in Anadolu University’s traditional distance education system. Some of 
the limitations of the system that supported this decision are that 

• There are too many students to provide two-way, interactive instruction 

• The programs requires individual, self-paced learning 

• There is not enough number of instructors 

• Lack of student-to-student interaction 

• Time limitations of the television programs 

• Page limitations of the textbooks 

• Cost-benefit factors 
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However, the experts indicated that some of the principles of constructivist view might be used in designing 
textbooks for distance learners of the Anadolu University. Providing realistic examples while presenting the content, 
including directions that encourage students to find their classmates live in the same area and study together, giving 
authentic problems to solve (or projects to complete or cases to analyze) as embedded tests, providing opportunities 
to generate solutions, ask questions, underline important points, presenting readings related to the content and 
suggesting a list of other information resources are the main principles that the experts offered to use in the design of 
textbooks. 

As a result of this phase, the textbook for “Introduction to Business Management’’ course, includes these 
principles for Anadolu University’s distance learners, has been designed and developed. Later this material 
presented to the experts to determine if the suggested principles properly used. Experts indicated that most of the 
principles somehow successfully incorporated but especially the one related to directing students to different 
information resources was not proper. Experts explained the proper way of suggesting other information resources to 
the content experts. However, they could not give the information resources properly and the textbook is published 
and distributed to the learners. 

At the last phase of the study, an evaluation questionnaire was prepared to determine the learners’ 
reactions to the textbook. The questionnaire was consisted of a Likert type scale including 38 items and a 2 open 
ended questions. The 12 of those 38 items were directly related to the parts of the textbook that reflect the 
constructivist principles. 

The questionnaire was sent to the academic advisors of AOE in seven different cities of seven provinces of 
Turkey. These cities were Eskisehir, Gaziantep, Aydin, Van, Canakkale, Samsun and Mersin. The advisors 
administered the questionnaires to the 457 Anadolu University’s distance students who had been participated their 
face-to-face evening classes. 

Erequency distributions and t-test were used in analyzing data. Eor the analysis of the frequency 
distributions, 3.41 mean score determined as the critical value. That is, the means equal and higher than 3.41 were 
interpreted as positive attitudes. Table 1 shows the frequency distributions, mean scores and standard deviations of 
the items related to the constructivist view. 

Table 1: Erequency distributions, mean scores and standard deviations of the items related to the constructivist view 



Item# 


N 


1(%) 


2(%) 


3(%) 


4(%) 


5(%) 


M 


SD 


2 


457 


21 (4,6) 


55 (12,0) 


57 (12,5) 


213 (46,6) 


111(17,5) 


3,7396 


1,1160 


4 


457 


26 (5,7) 


47 (10,3) 


108 (12,5) 


196 (42,9) 


80 (17,5) 


3,5624 


1,0704 


5 


457 


21 (4,6) 


70(15,3) 


84 (18,4) 


188 (41,1) 


94 (20,6) 


3,5777 


1,1160 


6 


457 


19 (4,2) 


41 (9,0) 


82 (17,9) 


209 (45,7) 


106 (23,2) 


3,7484 


1,0409 


9 


457 


21 (4,6) 


28 (6,1) 


74 (16,2) 


207 (45,3) 


127 (27,8) 


3,8556 


1,0392 


10 


457 


17 (3,7) 


28 (6,1) 


68 (14,9) 


203 (44,4) 


141 (30,9) 


3,9256 


1,0168 


II 


457 


21 (4,6) 


58 (12,7) 


86 (18,8) 


191 (41,8) 


101 (22,1) 


3,6411 


1,0974 


12 


457 


66 ( 14 , 4 ) 


79 ( 17 , 3 ) 


112 ( 24 , 5 ) 


149 ( 32 , 6 ) 


51 ( 11 , 2 ) 


3,0875 


1,2323 


15 


457 


13 (2,8) 


36 (7,9) 


58 (12,7) 


231 (50,5) 


119(26,0) 


3,8906 


0,9739 


17 


457 


21 (4,6) 


48 (10,5) 


93 (20,4) 


215 (47,0) 


80 (17,5) 


3,6236 


1,0358 


37 


457 


27 (5,9) 


41 (9,0) 


72 (15,8) 


169 (37,0) 


148 (32,4) 


3,8096 


1,1570 


38 


457 


17 (3,7) 


21 (4,6) 


55 (12,0) 


190 (41,6) 


174 (38,1) 


4,0569 


1,0093 



The results revealed that distance learners have mostly shown positive attitudes toward the constructivist parts of 
the textbook except other information resources part. They indicated “no comment’’ on this part of the textbook (M 12 



= 3.0875). The learners expressed that they like the real life examples most (M38 = 4.0569; Mjo = 3.9256). 



Eurthermore, the t-test analysis has shown no significant difference between means scores of female and 
male students. Also there is no statistically significance between mean scores of students who live in eastern part of 
Turkey and western part of Turkey. 
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Conclusion 



Does not matter how successful they are, all the educational institutions should always work toward 
improving their services. Anadolu University, as a mega university, always tries to do so. 

As a result of those tries, a textbook model for distance learners was developed by senior instructional 
designers of the University. The claim of this model is that some parts of the textbooks might reflect constructivist 
learning principles. 

A textbook titled as “Introduction to Business Management” for the distance learners of Anadolu 
University was designed and developed according to this model. The parts that reflect the constructivist view of 
learning are related to provide realistic examples while presenting the content, to include directions that encourage 
students to find their classmates live in the same area and study together, to give authentic problems to solve (or 
projects to complete or cases to analyze) as embedded tests, to provide opportunities to generate solutions, ask 
questions, underline important points, to present readings related to the content and to suggest a list of other 
information resources. 

Results of the evaluation questionnaire have revealed that all the students (female -male, live in eastern part 
of the country or western part) expressed positive attitudes on the parts that reflect constructivist views. They 
especially liked to come cross the real life examples rather than fiction. 

This effort and study show that even the non-traditional students have positive ideas about the 
constructivist principles and designers should look for more ways of incorporating principles of constructivist 
learning principles in any all kinds of instructional settings. 
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Training to Performance Support: A Case Study in Change Management 

Helen Gregory, Veena Mahesh, and Susan Tatby 

Intel Corporation 
Abstract 

This paper expands on a human performance technology model used by a training department within a large 
global corporation to help transition into becoming a performance-centered group instead of focusing entirely on 
training. The paper specifically addresses the change management principles and processes used to ensure a 
successful implementation of the human performance technology model. It also outlines benefits and results in 
addition to issues and challenges faced during the implementation of the model. 

Introduction and Problem Statement 

Make the shift from training to performance support! Everybody’s talking about it. Cost reduction and the 
need for revenue enhancement provide the driver to deliver products faster, better and cheaper (Tosti & Stotz, 2000). 
Although it’s easier said than done, a change in focus from training to human performance technology (HPT) needs 
an emphasis on enabling individuals and organizations to meet their goals by delivering appropriate solutions 
(Farrington & Clark, 2000). For that reason many training divisions within major corporations are trying to make the 
transition to becoming performance support organizations. This paper presents a case study on how a training 
department within a large global corporation approached this transformation by creating and implementing a new 
performance improvement process to address production, quality and speed of solution development. The new 
process, the Process Life Cycle (PLC) Workflow, takes advantage of emerging capabilities in training and information 
delivery technology while retaining an emphasis on supporting performance. 

In stating the original problem, we had to admit that we, like many training departments, had a well- 
developed order-taking process but were not experienced in the more subtle types of skills and practices that would 
facilitate our transition from our order-taking role to the new role of change-agent/solution-recommenders. We were 
comfortable with our order taking methods, and our customers were accustomed to it. Our own departmental 
performance analysis indicated that we needed not only to reset expectations across the department and within our 
customer base and foster partnership arrangements, but we also needed an established process, refined tools, skill 
development, a different level of management support, and systems to support the implementation of new ways of 
working. 

The Solution - Part One 

Upon reviewing the solutions that we could implement to enable us to make the transition to becoming a 
performance support organization, we concluded that a standard, repeatable process that incorporated documents, 
templates and guidelines for performance consulting would be the first solution recommendation. This decision was 
based on key learnings related to analysis and design gaps that were identified in some initial ventures outside of 
training. This performance improvement process (the PLC), helped integrate performance thinking into the everyday 
business workflow of training development. 

The PLC Workflow is a human performance technology, problem/solution methodology for enhancing 
human performance in the workplace. The model combines processes from standard problem-solving methodology, 
instructional systems design, and human performance technology. It stresses the importance of applying a stand- 
alone repeatable methodology to performance problems and opportunities. The PLC workflow was enhanced with the 
addition of specific elements of customer engagement as well as other business process elements required by our 
corporate environment and infrastructure. In addition, the process emphasizes and enhances a number of elements 
that were seen as important to a performance support model: Analyze Performance and Analyze Cause and an 
expanded Intervention Selection phase. 

Along with the PLC process, we developed a number of tools and templates to support performance. Tools 
were modeled partly on some of our own best practices and partly after a group HPT tools that were adapted from the 
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International Society for Performance Improvement (ISPI). We developed an extensive web site to make the tools 
more accessible and to further explain the use of each tool. The site structure is based simply on standard HPT 
phases along with additional sections to provide quick access and enhanced usability to the tools and templates. A 
search engine allows the user to access information quickly on any specific topic of interest pertaining to the PLC. 
The web site also provides access to a useful element of support in the PLC, the generic Gantt, developed in MS 
Project. The Gantt provides a checklist and timeline approach to the PLC process, with notes to help clarify steps in 
the process, hyperlinks to tools and templates, and clearly identified input, output, and dependencies for each step in 
the process. 

The Solution - Next Steps in Change Management 

“Build it and they will come” is a catchy phrase, and it might work in some arenas; but we found that 
changing organization and customer expectations involved more than making a process and tools available for use. 
Implementing a new work model and getting it adopted is a lengthy process and often requires alteration in the 
structure and function of the social system in which it is deployed (Day, Shoemaker, & Gunther, 2000). Change 
Management is also an essential component of the Implementation portion of the ISPI HPT model. For performance 
technologists, organizational change is a route to achieving their goals of productivity, cost reduction, and quality 
(Rosenberg, 1995). Organizational change is an area of professional practice that involves the proactive management 
of change using standard models, techniques and tools (Nickols, 2000). It usually results in organizational change, 
which is defined as follows: 

“Organizational change is a social process taking place over a period of time during which people and 

systems must undergo significant change, learning, adaptation and growth” (Carlopio, 1996, p. 1). 

The change campaign we employed involved the delineation of a strategic model for managing change, 
which included components such as organizational change, employee development, performance support and 
communication. Using HPT strategies within our own implementation enabled us to role model Human Performance 
Technology principles as well as provide critical support to the performer. This strategy focused our efforts on the 
most critical resource of our organization: the people. 

To implement the PLC Workflow and to enable people to follow the steps it outlines, we applied a number of 
key change management steps to our implementation to ensure that the process and the supporting tools and 
templates are fully incorporated into the day-to-day business process of the department. Training on the use of new 
process and tools, along with mentoring and coaching were used to support the process. We relied on gathering 
input from stakeholders at every step along the process development to ensure strong buy-in from all levels of 
management and employees. The involvement across all levels in the department was also a direct attempt to ensure 
full departmental ownership of the new processes and tools. 

The details around the actual steps involved in our change management roadmap are outlined in Figure 1 in 
greater detail. 
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Principles of 
Change Management 



People agree on the 
important things to do 
in the organization. 



Changes need to happen 
within the existing system, 
culture, values. 



Everyone can 
participate in some 
way. 

Management legitimizes 
the change through voice 
and action. 

Benefits of the change are 
clear. 

Cultural values are 
represented In real world 
practices. 

The change represents the 
views of the group as a whole. 

The change must be 
systematized. 

Change needs to be 
linked to more than one 
event 



Behavior of all groups is 
aligned and all are expected to 
participate. 

Change Management principles adapted 
From: “Large Scale Organizational 
Change," Donald T. Tosti, PhD. 



Figure 1. Training to Performance Support Change Management Roadmap 
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Ongoing Challenges 

To say that our implementation was a smooth transition would be a significant denial of the pain involved in 
this type of transition. With 129 people in our department, perhaps our biggest challenge was around communication. 
We found that we could not communicate too often with the different organization units in the group. We also found 
more resistance to changing the way we worked than we might have hoped from some of the late adopters in the 
department. Our early practice of involving all members of the organization needed to continue as an ongoing part of 
process improvement. Instilling discipline around thorough up-front analysis is an ongoing challenge that we hope 
will become somewhat easier with the implementation of an automated tool for parts of the business process. 

These challenges have mandated a continuing effort around implementation. And as more people in the 
department begin to embrace the change, their involvement, while a positive indicator of a successful implementation, 
makes it necessary to continue to monitor the process closely and to maintain a formal change control process. In 
tight economic times, maintaining the commitment from management for continued resourcing is a challenge that has 
become a seemingly constant factor. 

Benefits and Results 

In spite of ongoing challenges, the implementation of the PLC Workflow has resulted in a number of 
benefits for the department. Initial data regarding the implementation show a high rate of process awareness across 
the department when compared to earlier data that show a lack of process awareness. Initial data related to defect 
reduction showed a 50% decrease in defects rated important, 100% reduction in critical defects for web-based 
training, and an 82% level of user confidence that PLC adoption will lead to process standardization. In addition, 
process applicability has led multiple groups outside of our own department to pursue using the PLC more widely 
across the enterprise. 

We are currently at the one-year point following our initial implementation and we see some additional 
benefits to implementing the PLC. We see people in the department starting to use a common language: we now hear 
people discussing projects in terms of analysis, design, requirements, and testing. In addition WBT products built 
using the PLC are being released consistently with zero defects in integration testing, and we are seeing an increase 
from 100 to almost 200 hits per month on the PLC web site indicating an increase in users for the process in general. 
PLC use has resulted in products that receive customer kudos, and in our most recent employee performance 
evaluation cycle, the PLC methodology was mentioned as the means for achieving results. Lately we have had 
experiences with some late adopters requesting help for PLC components to be implemented into vendor equipment 
training. Several strategic directives have also focused on cross-functional alignment and optimization of the PLC. 

We have also made progress against our original goal of transforming ourselves from being a training 
organization to becoming a performance support organization. One of our department’s key strategic programs for 
2002-2003 is a large-scale Electronic Performance Support System (EPSS) initiative. We have made progress in 
supporting our automation customer base by becoming Graphical User Interface (GUI) consultants where we 
previously had often been asked to provide training to compensate for poorly designed application interface design. 
We are also developing methods for designing solutions around the delivery of multiple knowledge product types, 
and we have implemented a number of limited Knowledge Management (KM) solutions. When one of our partner 
organizations requested training to remediate the implementation of their own PLC, we were able to offer some of our 
key learnings around human performance technology as an alternative. 

Key Learnings and Recommendations 

Our key learnings from this implementation seem to fall naturally into two general categories; those related 
to people and those related to time. The fact that people are inherently resistant to change cannot be overlooked in a 
transition of this type. It might seem like an obvious statement, but acknowledging it openly, planning for it, and 
understanding the pockets of resistance within a given organization can be immensely helpful in understanding how 
and where to focus attention. Process change must be tied to specific environmental factors which requires a solid 
understanding of the organization and the environment in which it operates. Our own environment offered a number 
of specific challenges that were inherent in our culture such as the way we reward people. Those types of factors can 
in no way be generalized. 
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Another key learning related to the people aspect of change is that once you include everyone, you really 
have to include everyone’s ideas, and it is imperative to stay open for others to contribute to the change. We 
understood from the onset of the project and our early change management considerations that this aspect was 
critical. We may have underestimated the challenge this would pose, however. As more and more people accept the 
new process, more and more people want to be involved in defining and refining the process. This poses challenges 
around integrating the new ideas and sometimes around making decisions about which ideas to even include. It 
appears to be a complex process that requires ongoing orchestration. 

It is little wonder that our industry has been discussing the topic of moving from training to performance 
support for a number of years. Some of our own key learnings are related to the time aspect of change. To say that 
large scale change cannot be event based may be an oversimplification. Our experience indicates that we cannot 
overemphasize the need to think long term and that when you think it’s finished it’s really just starting. Although we 
planned and held project celebrations a year ago at the point of our initial implementation, we still face the ongoing 
challenge of integrating the increasing number of ideas that are added to the mix. Plan to resource this type of 
transition for longer than the initial estimate. Our department continues to resource this change implementation and 
will likely continue to do so for some time. 

Finally, it’s critical to appreciate the value of even small steps in the right direction. To undertake and own a 
transition of this type, it is essential to have vision and passion around making that vision become reality. Sometimes 
the level of passion required to support this type of change can result in impatience. It helps to temper that passion 
with a lot of patience. The understanding of organizational change being long term as opposed to event based really 
translates to the change occurring in small steps over a period of time. Understanding that and accepting it at the 
outset may help ease the transition. 

Future Plans 

The future resourcing of this effort within our organization will be centered on a number of planned small 
steps that will continue to move us toward our overall goal. We have immediate plans for additional upskilling in 
Analysis and Requirements Engineering. We continue to learn new ways to improve tools and templates connected 
with the PLC to make it easier for even non-experts to use them. To date our process is only fully developed for a 
number of training and software interventions. We would like to incorporate process definition for some of the 
knowledge-based solution that we are beginning to design and develop. 

Since one of our key drivers for this transition was related to cycle time decreases in our industry, we are 
working to develop tools for better tracking of cycle time by phase. We have identified a department metric around 
cycle time with the ultimate goal of using enhanced project tracking tools to achieve cycle time optimization. To that 
end, our department is at the point of testing a new tool to automate this tracking. 

Our long-term future plans go beyond implementing the change in our department. In response to requests 
from some of our own internal groups we plan to explore the feasibility of productizing the PLC workflow web site 
and tools and templates for use by equipment vendors. Productizing will also better enable us to share our process 
across the corporation. 

Summary 

The paper delineates a strategic model for managing change, which included components such as 
organizational change, employee development, performance support and communication. Key learnings from the 
implementation of a human performance technology model fall into two general categories; those related to people 
and those related to time. All experiences point to the fact that large-scale organizational change should be planned 
out over an extended period of time, must be inclusive of the people that it will impact and requires a good 
understanding of the organization and the environment in which it operates. Above all, it needs planning, passion 
and dedication in order for it to be successful. 
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Pulling Tigers’ Teeth Without Getting Bitten: A Robust Human Relationship 
between Instructional Designers and Faculty 

Cheng Chang (Sam) Pan 
Jennifer Deets 
William Phillips 
Richard Cornell 
University of Central Florida 
Abstract 

This directed qualitative research project concentrated on an instructional designer’s (ID) expert qualities and 
their inter-relatedness to interpersonal dynamics with faculty in terms of the role the ID is playing. The designer’s 
personal practical theories (PPTs) and their relationships to his effectiveness as an ID were discussed. 

Researchers hope to present a vivid portrait of an ID and to provide insights into the profession as reflected in a 
southeastern university setting. Implications and significance of the case study are also addressed. 

Introduction 

“Pulling tigers’ teeth without getting bitten”— a Chinese proverb, annotation 
of the accomplishment of a task that includes great danger or risk (Anonymous, 
n.d.). 

In cybereducation, a term used by Vandervert, Shavinina, and Cornell (2001), the role of instructional 
designer (ID) has arisen as a result of the increasing presence of web-based instruction. An ID’s role is unique, 
neither clearly leading nor supporting, as revealed in the comments of one faculty member: 

He makes appointments to suit my schedule ( rather than his), often within a 
very short and pressing timeframe. He takes the time to find out what I really 
need (rather than coming from presuppositions), then gives me concrete, easy- 
to-follow instructions. 

Making an appointment based on the other individual’s schedule is a course of action expected of a 
subordinate, whereas giving instructions is deemed the prerogative of a leader. Yet both actions were revealed in the 
above comments, indicating a need for more nuanced understanding of the role. Thus, this project was undertaken 
as an ethnographic case study in the hope to better understand the complexities involved in the work and experience 
of IDs. 

One of the researchers, Sam (as a student researcher), and participant ID, Bill, both work in Course 
Development and Web Services (CDWS) at the University of Central Florida (UCF). UCF has been implementing an 
online initiative since 1996 (have been using WebCT since 1997) and in 1998 won the American Productivity and 
Quality Center and State Higher Education Executive Officers (APQC-SHEEO) Eaculty Development Award for 
Teaching with Technology. The United States Distance Learning Association (USDLA) conferred the 1999 
“Excellence in Distance Learning Programming Awaid in Higher Education’’ on UCE for its use of distance learning 
media, including embedding the World Wide Web in online courses as an instructional strategy. 

CDWS was established in order to create a staff of IDs to work with UCE faculty members (subject matter 
experts) in the development of pedagogically sound design, implementation, and evaluation of asynchronous 
learning courseware on both one-on-one and team basises. The instructional design team is one of the eight units of 
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CDWS supervised by the executive team. The ID team is known for supporting UCF faculty teaching and student 
learning by means of consulting and facilitating training of the clients (UCF faculty members and/or graduate 
teaching assistants) as well as coordinating with the other seven teams of CDWS (University of Central Florida, n.d.). 
The uniqueness of any ID’s job is unquestionable; however, whether the ID’s role is leading or supporting remains 
less clear. Understanding the nuances of the roles that IDs play is a major issue that the authors highlight and 
discuss in this report. 



Background 



An ID’s job performance is strongly related to his or her experiences with real world situations and projects 
(Julian, 2001; Summers, Lohr & O’Neil, 2002). Professional IDs depend on prior experiences to perform their complex 
work on a daily basis (Julian, 2001). 

An ID is conceptualized as a teacher, based on the nature of the ID's job -- conducting workshops, teaching 
groups for various purposes, and holding customized consultations. Frequently these consultations are to facilitate 
faculty members' and Teaching Assistants' (TA’s) literacy of educational technology to help them meet their 
instructional needs. Sanches (1993) also argued that “the teacher appeared as decision-maker, ‘rational executive,’ 
and as physician who diagnoses learning needs and prescribes ‘instructional treatments”’ (p.23). The authors 
wondered what and how the ID's previous experiences might affect his beliefs about his job, his instructional 
decision-making, and his effectiveness. 

The relationship of an individual's previous life experiences and activities to his or her beliefs about 
instruction as a result of designing, implementing, and evaluating the curriculum can be emphasized in the notion of 
personal practical theories (PPTs) (Cornett, Yeotis & Terwilliger, 1990). Cornett (1990) stated that each teacher 
possesses PPTs, which are statements of what he or she believes with respect to decision-making in curriculum and 
instruction. Cornett stressed that these PPTs are derived from all life experiences, non-teaching as well as teaching 
activities. Grimmett and Erickson (1988) argued that PPTs are not merely practical beliefs of teaching but a type of 
ideology that drives the individual to more consistently reflect on his or her perceptions, beliefs, and practice. 
Through reflective practice, theories are tested repeatedly; judgments and beliefs are built. Reflection is essential in 
order to better understand educational experiences and to develop better curriculum and instructional skills, 
knowledge, and attitude. Reflection is also a recurring theme in teachers' sense-making, concept development, and 
decision-making (Ertmer & Quinn, 1999; Rowland, 1992). 

These reflective activities are in concert with what Condit (1996) coined “critical common sense.” Condit 
stated, “This common sense is a reflective one, one that takes into account not only the lessons of daily experience, 
but also reflection on how those experiences fit into larger social structures, [thus] one can refer to “critical common 
sense” (p.l69). Critical common sense about teaching is reinforced and accentuated by continuous interaction 
amongst professionals working within unique organizations with unique situational factors affecting the interaction. 

Situational factors, such as an organization’s culture, structures, and values (Erost & Teodorescu, 2001), 
exert a significant influence on professional competence and practice (Kaufman, 1990). Environmental attributes or 
external forces affect any worker's motivation, professional competence, and job performance if a supporting 
institutional culture is not created and nurtured in response to inevitable change (Votruba, 1990). 

Choosing to focus on the motivation dimension of external vagaries of organizational life, 
especially as articulated in Herzberg’s (1966) work motivation theory (the motivator-hygiene theory), two concepts 
are cmcial: basic needs, such as salary, peer relationship, supervision, and company policy, and growth needs, like 
achievement, recognition, advancement, responsibility, and work itself. In Herzberg’s language, basic needs are 
hygiene factors, and the lack of those factors is associated with job dissatisfaction, whereas the presence of growth 
needs, which are also named motivators, would attribute to the feelings of growth and development at work. 

Although debate about generalizability, oversimplification of Herzberg's theory persists (Cooper & Locke, 
2000; Earr, 1977; Graen, 1966; House & Wigdor, 1967), basic needs and growth needs are useful constructs for this 
study. Eor example, two growth needs, work itself and organizational processes, are also two vital sources of 
motivation (Earr & Middlebrooks, 1990). As Finder (1998) suggested, “One need only believe that building jobs to 
provide responsibility, achievement, recognition for achievement, and advancement will make them satisfying and 
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motivating” (p. 38). Further study for more empirical support is also recommended by Brief (1998). Thus, how the 
ID’s basic needs and growth needs interrelate with his PPTs and, further, how they affect his performance at work are 
major factors in understanding the nuanced role(s) of the ID. 

Research Questions 

Following the five elements of the case study design by Yin (1994), the authors proposed three research 
questions: (1) What is the nature of the role(s) played by the ID? (2) What is the relationship of the ID’s role to 
interpersonal dynamics with faculty? (3) How do the ID’s basic and growth needs interrelate with his PPTs? 

Methodology 

In addition to Yin’s (1994) case study design, Tellis (1997) recommended three major components of 
qualitative method that need to be employed: describing, understanding, and explaining. The ID was selected based 
on familiarity, which smoothed the student researcher's transition from simply a colleague to a participant observer. 
Additionally, asking a familiar colleague to serve as research participant expedited entree, which was a consideration 
given the short duration of a typical academic semester. The collegiality between the student researcher and the ID 
smoothed the process of data collection. 

In addition to interviews with the ID and observation of him alone and in interaction with others, a seven- 
item online survey was designed to target 70 colleagues and clients, who had or have been working with the ID in the 
past two years. Although the response rate was not strong (approximately 26%), 18 valid respondents (thirteen 
faculty or staff, four coworkers, and one unidentified) shared their perceptions of an ID's role, especially Bill's. Video- 
taped interviews with the ID and five other ID’s (including one of the supervisors from the executive team) were 
conducted. The videotape was transcribed. The student researcher was scheduled to spend three hours each week 
in the field (i.e., the ID’s office) observing the ID during the first eight weeks of the spring semester in 2002. E-mail 
correspondence between the ID and 25 of the ID’s clients were chosen, and the results reviewed and analyzed. 
Information on the unit’s and the Human Resource office’s Web sites was documented. Copies of official job 
descriptions, used from the last round of ID recruiting, were provided by the ID, photocopied and filed. The goal of 
such description is to assist the reader in knowing what the researcher saw by visualization and emotion (Eisner, 

1998; Glesne, 1999). 

Regardless of the evolving themes in this case study (Zucker, 2001), the iterative process of analysis could 
be considered a sort of triangulation as it was intended to enhance the worth of the project (Kaulio & Karlsson, 1998; 
Silverman, 2000; Yin, 1994). Eor pragmatic and instructional purposes, the faculty researchers confronted and 
questioned the student researcher to refine and distill the latter’ s understanding and thoughts about the case as well 
as to reconsider and review his judgment skills. 

Findings of the Study 



Roles 

The dynamics between the ID, Bill, and the faculty are intimate but vague in terms of interpersonal dyads. 
Due to the nature of an ID’s job. Bill works closely with clients, especially when IDL6543, an eight-week-long faculty 
development course for online teaching and learning begins. While he was facilitating a first-time faculty member's 
use of WebCT software, an interesting dialog occurred: 



Bill: 

Faculty (FI ): 
Bill: 

Faculty (FI ): 



You went to the GroupWise thing [a training session on the use of the university’s E-mail 
program] in the morning? 

No. 

OK. Before you leave, remind me, I have the handout from that session. This Friday we 
are gonna teach you how to set up what is called ‘rules’ in GroupWise. 

Oh, OK. 
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Asking for a reminder like this was not a breach of professionalism for the ID. The question indicated that 
they work closely with each other, so the ID did not feel uncomfortable about asking for information like that. When 
it comes to faculty’s preference for a varying degree of service from the ID, the following conversation occurred: 



Faculty (F2): 
Bill: 

Faculty (F2): 
Bill: 



I think it’s [giving individual consultation] helpful, myself. Do I have the ability to go in 
and make changes to the REACH pages ( i.e., WebCT course pages)? 

Yes. Yes. 

O.K. I didn't know that. That's why I stopped using it, because I thought I had to go 
through the (instructional) designer to set those up. 

You can use us, or you can do it yourself. . . 



For this first-time faculty user of WebCT, a close relationship between him and the ID is evident in this face- 
to-face individual consultation. “Use” is causal language as indicated from the previous conversation. In this case, 
the ID reminded the faculty member that he could do the work alone or could use the services/assistance of the ID 
staff. Bill indicated that the faculty member’s preference for one route or another did not matter to him, but that he 
was willing and glad to be helpful as needed. The consultation also clarified some misunderstanding of how the ID 
functions. 

The authors use the relationship between a catcher and a pitcher as a metaphor. The pitcher takes the 
catcher's (usually the team captain) hand signals to throw different curves, and then the catcher uses all means to 
catch any ball the pitcher threw. Based on the closeness of the ID/faculty member relationship, the authors believe 
that the relationship established in the creating and maintenance of on-line courses was as partners and teammates. 

The relationship between the two players - ID and faculty member - was strong and interdependent. The 
ID’s primary job is to design, in WebCT, and to set up courses on the Web. Most faculty interviewed perceive the 
ID's job to be technical-knowledge-based rather than pedagogically -oriented. Although the consultations observed 
by the student researcher were related to technical questions and issues, a Web-based course will not exist without 
knowledge, skills, and right attitudes from the two parties. They both influence each other. 

Bill: But if you ever have a question for me, I am gonna be working with you and developing a course. 

The best way to get me is email. Or if you want to call here, you may want to try and see if I am 
around. What I am gonna do for you is to get you a good course experience. 

F2: There is something that I find, something really amazing, about developing courses through 

technology. Students walk in the classroom and interact in the discussion board. They become 
more active. 



Indeed, for a novice on-line instructor, a good beginning experience can affect their attitude toward using 
the system and a good experience ultimately affects the actual use of the technology (Harris, 1999). Like fish and 
coral reefs that depend on each other for survival, the ID and faculty members in a sense cohabit within a closed 
system. Without the technical and pedagogical knowledge from the ID, a course with good content (from faculty or 
subject matter experts) will not succeed on the Web. 

As previously mentioned, the ID always made an appointment at a suitable time to conduct an effective 
consultation; meanwhile he exerted an influence on faculty by giving constructive advice. He acted as a leader who 
offered suggestion and guidance, but he was also adaptive and willing to show his support to maintain a balanced 
relationship. A client recalled: 

Bill is easy to talk to. He is prompt in responses to any questions I have had. He 
is knowledgeable about WebCT and keeps me abreast of changes. He is a 
pleasant person to work with. In no way does he condescend. He never brushes 
me off— even when Fve done something stupid. 
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On the other hand, if the shared goal is not sustained (i.e., designing a sound Web-based course), faculty 
can maintain good quality of instruction by teaching a conventional manner (i.e., face to face instruction) or they can 
teach “whatever and however they want’’ on the Web. Another conversation between Sam and Bill is as follows: 

SR: What do you think of this class? A class without a sound instructional 

design... 

Bill: Right, and I am gonna tell you... I am gonna recommend to that 

instructor what he should be doing to really kind of hold the line to make it 
right, but he is not gonna listen to me. ..and that is fine. If what he is doing 
makes him successful, makes his learners successful, then he is doing something 
right. Maybe that is a little bit radical for a real instructional designer to talk 
like that. If it works for him, it works for students, then it works for me. 

Bill’s stance in this case, was strongly in favor of neutrality on his part. For IDs, if the mission (i.e., 
supporting teaching and learning at the university, affiliated academic units, and partner organizations) is not carried 
out, their presence and existence will not be justified. One of the practices this ID honed was developing and 
maintaining a smooth relationship with his clients. The need for smooth relationships also holds true in corporate 
settings. Liang (1999, p.319) claimed that “They [IDs] present a low profile, a technique helpful in developing and 
maintaining personal relationships. They may sacrifice professional ego to accomplish managerial and people- 
oriented tasks.’’ Similarly, when the faculty member’s experiences and competencies of teaching online courses 
increase, IDs will be of more of an assistance role rather than an advisory one. 

Eventually, both IDs and faculty become more field independent, which is what Kelvin Thompson, 
Coordinator of Professional Programs of CDWS at UCF described as “three ways to see the relationships playing 
out, very simply, in terms of control, authority, and power hierarchy or clout.’’ He saw one with the ID being 
subordinate to the faculty member, another with the faculty member being subordinate to the ID, and the final one 
being one of balance. In an interview he said, “The third way I perceive kind of playing out is one of the more 
independence. There is alongsideness, coming alongside, being supportive and helpful, you know, something of a 
partnership.’’ 

Personal Qualities 

Whether working in corporate journalism or for the university, the ID is an individual who is enthusiastic 
about what he is doing and seeks fun in his work at all times. Fun is the source of his motivation for work. 

According to the people around him, the ID appeared to have personal traits that made him stand out from his peers. 
After a workshop at another campus, a university staff person told the student researcher 

Staff I (SI ): He is casual, he is very relaxed, and he is humorous... kind of 

takes the stiffness out of the way... he continued to go over his points and 
instructed that you have to do something and kind of gives you a recap what 
we have done. 

The ID was described by his colleagues as a professional (i.e., knowledgeable about what he does), who 
was personable, reliable, and humorous. A coworker said on the survey, “Again, I think his humor is what makes him 
great to work with. He always gets his job done but always in a light-hearted way. Bill is awesome.’’ Another 
indicated, “His sense of humor puts people at ease which allows for easier communication.’’ The other mentioned his 
great communication skills and that the ID "works/plays well” with others. 

These co-worker comments were congruent with how his clients felt about him. Their impression of the ID 
was that he was a patient professional with an appropriate mindset. A faculty member wrote on the survey: 

He's intelligent and has a sense of what faculty do. He is open to letting faculty 
deal with technology themselves rather than forcing us to submit to the whims 
of Course Development [where the ID works], who as a rule, seem to know 
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nothing of what faculty actually do. He has a sense of what good pedagogy is 
and cares that we are able to be the kind of teachers we are, despite what 
WebCT would seem to enable or disable. 

The quote also reveals that uniqueness and advantage of his personal traits and prior experiences as an 
adjunct faculty are involved in his job performance. While cultivating positive, productive interpersonal 
relationships with faculty, IDs in a higher educational setting often serve as change agents. As previously quoted, a 
faculty member commented that Bill has kept her “abreast of changes,” Thompson concurred on the “change” issue 
that ideally an ID should carry two dispositions, high tolerance of change and willingness to be the agent of change. 

When asked about the qualities of a competent ID at the university level other than what was addressed 
above, other IDs mentioned: organization (e.g., time management), empathy, assertiveness, work ethics, good energy, 
creativity, proactivity, interpersonal sensitivity (Snodgrass, 2001), and the concept of group-as-a-whole (Ringer, 
2002). Equally revealing are attributes of instructional technology professionals found in the AECT book. 
Technology in Instruction: Standards for College and University Learning Resources Programs, 2"‘‘ Edition 
(Cornell, 1989), in which almost all of the qualities for instmctional designers found within the UCE Course 
Development & Web Services are showcased. 

Needs and PPTs 

The ID provided the following list of his PPTs (Personal Practical Theories). The ten beliefs are listed in 
order of priority to the ID: 

1. Lifelong learning is essential. In order to prosper, mature or gain knowledge throughout life one must 
learn and continue to learn or become stagnant. 

2. Anyone can learn if motivated. Learning is fun if motivated and one has a desire to gain knowledge. 

3. Everyone learns in a different way. Each person learns in his or her own way and at their own pace. 

4. Learning must take place for one to grow. To grow on the inside or on the outside one must learn. 

5. Learning comes faster when doing while learning. If an exercise is handed to the learner, the learner will 
learn faster when doing as opposed to just listening. 

6. Adult learners are motivated learners. Adult learners want to learn and to grow and to gain knowledge. 
To the adult learner, knowledge is strength. Adult learners understand the consequences of learning or 
not learning. 

7. Learning in a group environment is stimulating. Playing off of others while learning is fun and 
stimulating and motivating for the learner. 

8. Learning should be organized in order to be effective. By organizing curriculum, one is facilitating the 
learner and makes learning easier to understand and absorb. 

9. Learning takes place at an individual pace. Each person learns at their own pace and in their own way. 

10. Learning should be in bite-sized nuggets. Too much curriculum at one time is not fun and not 
stimulating for the learner. Curriculum should be organized in bite-sized chunks to be learned 
effectively. 

While he was teaching as an adjunct, these beliefs started to grow in his mind: 

SR: While you were a journalist instructor. . . 

ID: Right. And I loved it, it was a fun course... students wanted to be 

there... wanted to learn as much as I could give them. To learn about 
communications and education. So we had a great time. And I discovered 
during that 7 years, that you know... I think... I was meant to be a teacher. So I 
kind of went in and learned a little more about being a teacher. 

Lrom the dialog above, “Students wanted to be there. . .wanted to learn as much as I could give them.” 
corresponds to his PPT6: Adult learners are motivated learners. The last sentence represents the PPTl: Lifelong 
learning is essential. Lor the ID, it is never too late to learn. In the interview. Bill talked animatedly about learning 
how to do some household projects. Bill is also pursuing an Ed.D. in Curriculum and Instmction at UCL, indicating 
his life long learning. To him, “fun” is the drive, and excitement is his motivation. In a sense, learning about being a 
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university teacher is not just advancement to him, but also a responsibility. Both advancement and responsibility are 
regarded as growth needs in Herzberg’s (1966) language. However, the coexistence of both growth and basic needs 
is identified in the following conversation: 



SR: OK. Tell me something more about the transition [from being in 

photojournalism to becoming an ID]. 

ID: Yeah, it [being an ID and teaching] was the only motivation. It was 

most of it. It is funny. I wanted something new, something challenging, 
something on the cutting edge, something close to home... 

The following conversation falls into his PPT2: Anyone can learn if motivated. 

SR: Correct me if I am wrong. ..you were saying if the course is fun, 

students would get motivated. Right? 

ID: Right. I think so. I think if it is fun. . . they will enjoy it. They are gonna 

learn more and they are gonna have a better time taking that class. Their 
grade may... you know... show through that. 

The ID set a goal that students come to his class and enjoy themselves in the class, which is in concert with 
another growth need, achievement. 

SR: You mentioned that maybe this summer you might have more time to do 

your personal learning profile thing? Is it required to do this? 

ID: We just actually formalized this. We are in the process of writing those 

now. That is, what do we want to learn and what would the benefit of that be to 
the unit? I kind of came up with an idea. ..last year we did IDA7000, 

Instructional Designer Academy...! am sure we are going to move it to another 
level pretty soon. But we created IDA and a couple of us went on the road and 
presented this. It was incredibly successful. IDA is basically a WebCT course 
where we teach instructional designers what we do and how we do it. I want to 
do the same thing with a course for adjuncts. Because I was an adjunct, I know 
the frustrations and I know what adjuncts go through. ..to find a parking place 
at night... you know, little things like that. ..I want to build not just a course 
that will teach adjuncts of all these logistics over here but also I want to give 
adjuncts. ..there are a huge number of adjuncts who teach at UCF...I want to be 
able to give adjuncts an online tool or an "E" course to facilitate their 
learners ' growth and make their teaching more effective. 

Based on the dialogue above, his PPT9, “Learning takes place at an individual pace.’’ was identified. An 
online training session occurs at an individual pace. At the same time, the work itself brought achievement and 
recognition to him and his abilities. 

He even used himself as an example when it comes to the notion that everyone learns in a different way (i.e., 
PPT3 and PPT5). 



SR: Do you believe that each of us learn differently? 

ID: I do. Because I am a visual learner. And I cannot tell you how 

frustrating it is for me to be given verbal information from an instructor and 
expect to learn it. So I need a picture. Maybe that is because my 25 years in 
photojournalism. I need a chart or picture or a diagram or a table. I am 
having a real hard time right now with a class I am taking, trying to 
differentiate between the behaviorist and constructivist... you know. ..Piaget 
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and Vygosky and you know... because I don’t have the pictures of all 
these... actually I know where the picture is. It is from a class I took last 
semester. It is in one of the textbooks. I just didn ’t go back and copy this to be 
able to visualize it. Therefore, when I teach, I teach toward the visual learners. 

I try to get the visual learners as many pictures as I can. And I am a firm 
believer of giving the learners something to do... if you involve them and create 
the interaction between the learners and materials or the content. I firmly 
believe that they will learn quicker and easier and comprehend more. 

He strongly believed that for various learners, different teaching models are needed. Web-based instruction 
(WBI) is believed to bridge the gap between these two factors (i.e., various learners and different teaching models). 
Kahn (1997) defined WBI. . . 

Web-based instruction (WBI) is a hypermedia-based instructional program 
which utilizes the attributes and resources of the World Wide Web to create a 
meaningful learning environment where learning is fostered and supported (p. 

6 ). 

Bill’s PPT5 is about learning by doing, which is illustrated in the following quote: 

SR: Is that what you do to your clients when you are doing your IDL and 

other training sessions? 

ID: Right. You don 't know how hard it is to do, because especially like I 

want to teach somebody how to dial in on Dreamweaver and change their 
office hours on their homepage. ...I can do it like that ( easy). Instead I want 
them to turn on the computer and open Dreamweaver to set up the dial in 
information, to dial in and connect to the site, and to make changes and save it 
and put it back up... I want them to learn how to do it but it is frustrating 
because I know I can do it so much quicker. But if they do it a few times they 
are gonna learn how to do it. If I do that for them, it is all about teaching a 
man to fish... 

His bite-size -nugget learning (i.e., PPTIO) is exactly what the research demonstrated is a productive 
approach to take. Chunking is a fundamental instructional tactic familiar to educators and serves to decrease the 
limited cognitive load of human beings (van Merrienboer, 1997). 

Workload 

The authors expected that external forces like work itself and environmental change could at times prohibit 
the ID from acting on his stated PPTs, and thereby interfere with his job satisfaction and possibly his performance at 
work. Although most of the PPTs the student researcher scrutinized were well-articulated with his personal growth 
and survival needs, one issue, workload (i.e., a large number of clients as well as additional workplace 
responsibilities), which is commonly seen as problematic in the organization, commanded attention (Mouly & 
Sankaran, 1999; Tattersall & Morgan, 1997). One of the faculty members who completed the survey pointed out an 
intriguing phenomenon, “He should have fewer faculty that he's responsible for or more assistants.’’ 

The ID stated: 

Since last December, after IDL and the Winter Workshop was finished, the 
faculty were knocking down our doors asking for our professional services and 
technology (WebCT) facilitation. Our workload has been increasing even 
since. We created over two hundred new courses in just over a month starting 
about December 15. The Pegasus Disc, faculty training, and faculty projects 
like Winter Workshop and Summer Institute and things like that kept us very, 
very busy. I can 't spend as much time on each course as I would like to. I have 
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ideas that can make these courses better in terms of sound pedagogy, but due to 
the time constraints...! feel bad about what is going on, because now lam 
always looking for shortcuts. I do not really mean that in a negative way but...! 
didn 't do that before. I do wish to spend more time interacting with faculty 'as 
they teach' f2f [abbreviation in the e-mail] and online... observing, 
challenging. I wish I could research other courses, schools and programs for 
good ideas. I just don 't have time to do it now. Hopefully the summer semester 
will allow us more time to think and write and work on some creative projects. 

Not only the faculty members, but the ID himself realized that workload may have exceeded his capacities. 
Thompson acknowledged this challenge in the interview: 

Obviously, we cannot keep the same number of people and continue to increase 
the workload. You cannot continue to pile on. Vke have a policy here that the 
faculty are free to be anywhere they choose to be on a full service to self service 
continuum with us... Most faculty are somewhere between those. ..those two 
extremes. That is all good. That is all fun. But that means if you worked with 
me once three years ago, you are technically in my workload. 

It sounds straightforward that the ID was drowning in his work. The student researcher wondered how long 
this situation could last before problems occur. If the ID failed to accomplish things that he used to do, did this 
change impact his work effectiveness and job performance in the faculty’s eyes? According to the survey results, 16 
out of 18 valid respondents indicated that the ID was weighted 5 (5 as the highest and 1 as the lowest), in terms of his 
job performance. One gave him a 4.9, and the others a 4. On average, the ID’s work effectiveness seemed not to be a 
problem to the clients. In any event, the student researcher suspects that the silent non-respondents (approximately 
54) may have a different viewpoint regarding the ID’s effectiveness at work, but due to the timeline of this study, the 
next phase of this study will have to address this issue in more depth. 

Conclusion 

The relationship between the ID and his clients (i.e., faculty) is subtle. While the interpersonal connection 
was reciprocal to some extent, the ID, in most of the work performed, was supporting faculty in teaching and learning, 
both from faculty’s viewpoints and what the student researcher observed from the university’s description of 
expectations, regardless of the leading role in the process of curriculum and professional development. It is, after all, 
a harmonious relationship in most of the cases. It is also the ultimate form of partnership and collegiality. Both the 
ID’s distinctive personal traits, such as humor, humanity, patience, and empathy, and his professionalism led him to 
be a success at work. These good qualities are also consistent with the four competencies of instructional designers 
at CDWS: faculty interface, curriculum development, instructional materials design and development, and curriculum 
delivery. The qualities the ID possesses are endorsed by Thompson: 

We have these two things working together. It is easier to characterize one 
curriculum development as an introverted trait and the standup training stuff 
as an extroverted trait. . . We expect people [ new ID applicants ] to do both. We 
often vote people out for that [not having both traits]. 

Additionally, Bill’s previous working experiences contribute implicitly to the formation of his personal 
practical theories as well as to his job performance. As discussed, his PPTs were interwoven with personal needs for 
survival and growth. All of these traits illustrated the ID’s life at work. Sources of data revealed that he essentially 
plays a sensitive but tricky role, as pulling tigers’ teeth without getting bitten. This finding may or may not introduce 
a vertical dyad relationship, which is a superordinate-subordinate relationship. A mutually -important, peer 
relationship was witnessed in this case. That this ID was able to interact well with a large number of differing 
personalities does not necessarily mean that a harsh or unbalanced linkage (perhaps fueled by perceptions of 
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differential professional status as determined by degrees held, etc.), suspected by the researchers, would not take 
place. According to Bode (1999), the issue of collegiality and mentoring to the new faculty is a concern of 
community building in the university setting. 

As the university continues to grow, more new faculty will be hired, some of whom will want to use on-line 
options for instruction, and the matter of the interpersonal skills of the ID to those newcomers remains considerably 
significant. Exploration of possibly unbalanced relationships would further the research in the next steps. Any 
possibility of an ethnographic case study of the ID team by conducting a 360-degree performance appraisal (i.e., 
collecting information from all directions and sources with respect to the ID team’s performance) can shed additional 
light on job performance of IDs in an educational setting. Application of the Western mindset in differing cultures 
with respect to instructional designers’ dynamics with faculty is worth probing (Pan, Tsai, Tao & Cornell, 2002). 

Over the years, whereas the ID team at UCF has gone through group socialization, it has also moved to a certain level 
of bureaucracy, which might hamper the ID’s individual job performance and, ultimately, degrade team effectiveness 
(e.g., creativity and tolerance to change) (Brown, 2000; Levi, 2001). This balance between the prevailing bureaucracy 
and the implementation of innovative uses of instructional design is also an issue of interest to Thompson: 

But I think as we have grown more established in ID [as a] team as we have 
more process and procedures, documents, and the trapping of bureaucracy...! 
am curious myself to what degree that tolerance to change is still there with 
individual or the team as a whole. I think within an individual primarily it is 
there, but with a team as a whole I wonder if it is there. 

After all, this case study documents, in detail, the robust relationship of the ID and faculty by examining the 
ID’s personal traits, needs, and philosophical beliefs of teaching. The paper can help other IDs with in-depth 
insights in better serving their clients in a time -efficient fashion, especially in the context of higher education. The 
value of this case study lies in its identification of the significant part an ID can play within an academic unit. It also 
provides university instructors with a closer look at the potential relationship and interaction with IDs. Moreover, it 
can allow university administrators to further examine how their systems might benefit from the UCF model and 
experiences, like job (re)design, resource allocation, and ID recruitment. 
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Introduction 

My intent is to develop an instructional theory to assist Self Directed Learners (SDLs hereafter) to learn how 
to monitor their learning process and develop needed skills to successfully reach identified learning goals. The 
purpose is to design instruction that makes learning painless, efficient, and a truly enjoyable experience for the 
learner (Reigeluth, 1999 [online]). One of my objectives is to assist SDLs to identify what methods of instruction to 
use in different situations. I intend to do so by creating a knowledge base about methods to support SDLs in their 
quest for independent growth and learning (Reigeluth, 1999 [online]). 

Goal and Preconditions 

The primary goal of the Personal Relationship Development (PRD) theory is to assist SDLs in learning to 
establish, develop and nurture healthy patterns of communication in long-lasting, meaningful personal relationships. 
The purpose is that SDLs take responsibility for structuring their learning process. A way to do this could be by 
setting expectations, designing assessment, facilitating practice and feedback opportunities, transferring performance 
skills into new situations, and evaluating their performance. 

PRD theory is specifically designed for youth and young adults entering long-lasting, meaningful 
relationships; independent adults who are new partner(s) in personal relationships; and for recently divorced adults 
who are starting new relationships. In my experience, I have learned that there is not a formal education forum for the 
general public that instructs how to develop communication skills to nurture and maintain healthy personal 
relationships. Therefore, I am targeting this audience with the objective to assist them in their journey to strengthen 
and improve their performance, participation and expectations in their personal relationships. 

A secondary audience could include families, counselors and advisors (e.g. ministers), classroom teachers 
and professors . Criteria that support the identification of this secondary audience are based on the access these 
individuals have to formal education settings. For example, if a teenager were exposed to structured mentoring and 
support environments that explored personal needs, strengths, and interests during his or her academic development 
years as a preventive approach to developing healthy personal relationships, I believe that a greater number of adults 
in society would be more satisfied with their performance and expectations of personal relationships in their lives. 

In conclusion, PRD theory has been developed with the purpose to assist SDLs in their quest to identify, 
design, develop, nurture and strengthen meaningful, long -lasting relationships that could function as their support 
system through life. I believe instructional support is highly needed in this area of human development and I have 
designed the PRD theory as a first step toward addressing this need. 

Principles and Values 

Some of the principles and values upon which PRD theory is based include the following: 

People learn at different rates. Most formal academic environments function under an old paradigm of 
learning that believes an individual learns a fixed amount of content in a fixed amount of time efficiently and 
successfully (Reigeluth, 1999). This is why most school systems continue to support and implement assessment 
tools such as standardized testing. PRD theory will try to reconcile the need to recognize and praise the abilities 
developed by SDLs as a new paradigm emerges, moving away from standardization into customization (Reigeluth, 
1999). 

Different instructional methods are needed in different learning situations. I believe that SDLs must be 
responsible for identifying best instructional methods for particular learning situations: cognitive learning domain. 



169 




affective learning domain, motor learning domain. PRD theory aims to provide a knowledge base to help SDLs in their 
quest for independent growth and learning. 

Personal Motivation should be the driver for learning and sustainability efforts. I believe that SDLs should 
develop a drive to pursue growth and learning through a lifetime. A key element in sustaining their independent 
efforts to succeed in set personal goals is their motivation to achieve them. PRD theory aims to provide structure to 
help SDLs maintain a high level of motivation through goal achievement. 

Careful selection of learning topics should facilitate understanding and transference of knowledge into new 
situations. Understanding happens when the learner makes/creates relationships among elements of knowledge. By 
constructing these relationships, knowledge elements are organized into structures, called schemata (Reigeluth, 

1999). I believe that when SDLs link constructed schemata, new learning occurs enabling them to transfer previous 
knowledge into new situations. PRD theory assists SDLs in constructing schemata inductively. 

Contribution 

PRD theory provides an inductive approach (from real situation to theory) to structuring, analyzing and 
successfully learning to establish patterns of communication in meaningful relationships. It is a constructivist 
approach to relational development. The latter is an utterly important aspect this instructional theory has to offer. 

Throughout our academic and individual development within our families and social groups, we have mostly 
been taught to relate to others under a Behavioristic approach. 

For example, growing up we have been taught to interpret that if a friend talks to one every day that means 
the relationship is healthy, but if the friend stops talking to one for a period of time that must mean the relationship 
has broken (Stewart, 1990). A behavioristic approach to human relations engages individuals in judging attitudes, 
emotions and behaviors toward relationships, both inwardly and outwardly. In 1986, Albert Bandura [Self-Efficacy 
Theory] advanced a view of human functioning in which an individual is viewed as self-organizing, proactive, self- 
reflecting and self-regulating rather than as a reactive organism shaped by environmental forces or driven by 
concealed inner impulses (Pajares, 2001). 

PRD theory gives attention to the possibility of entering and developing long-lasting, meaningful 
relationships from a Constructivist stand point in which all involved parties purposefully contribute to the 
construction of schemata resulting in a constantly novel, authentic and self-renewing relationship. 

Literature Review 

PRD theory is the product of a number of instructional theories. It came to exist based on a perceived need 
to address relationships and human interaction from a unique perspective. In order to understand how this theory 
functions and its implications, we must first identify the knowledge base that supports its structure. 

Self-Directed Learners 

According to Robotham (1995), master SDLs demonstrate ability to identify learning needs, have a positive 
self-image about their achievement capacity, know how to set and attain goals, have expertise in selecting or devising 
new strategies for different situations, are self-motivated and self-disciplined, are flexible in setting objectives, know 
their learning strengths and weaknesses, and demonstrate knowledge and skill in learning (Robotham, 1995, p. 6). The 
success of SDLs in establishing and developing healthy personal relationships is dependent of each and all of these 
characteristics. The more ready an individual is to direct his/her own learning, the higher degree of success in 
attaining his/her goals. All individuals are potential SDLs. The more guidance (vs. imposed control) received during 
developing years, the greater the number of self-directed skills attained/mastered. 

According to Lowry (1989), one way in which an individual demonstrates self-directedness is when s/he can 
be in control of learning topics, resources, content, and assessment (Oswald, 2001, p. 6). Grow (1991) developed a 
model of self-directed learning (SSDL - Staged Self-Directed Learning). He believes an individual moves from a 
dependent to a self-directed stage in his/her instructional decisions. Stage one learners (dependent) require coaching 
with immediate feedback; stage two learners (moderate) are interested and responsive to subject matter but ignore 
content; stage three learners (intermediate) are pro-active in their educational experience; stage four learners (self- 
directed / independent) take responsibility for their learning, direction and productivity (Grow, 1991, p. 134). In PRD 
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theory, SDLs are encouraged to move through different stages of performance depending on specific learning goals 
they are trying to achieve. SDLs must be in control to choose next steps in their learning journey. 

Garrison proposed that learners should have choices, flexible pacing and multiple resources in their learning 
process. He suggested that when an individual is in control of external conditions to his/her learning, s/he actively 
makes learning meaningful by monitoring his/her own learning process (Garrison, 1997). It is based on Garrison’s 
proposal that learner choice and control of learning situations are values of PRD theory because they increase 
intrinsic motivation. SDLs are responsible for maintaining high levels of motivation that can carry them through their 
efforts as they accomplish goals. 

Self-Efficacy 

Self-efficacy is the belief in one’s capabilities to organize and execute the sources of action required to 
manage prospective situations. Self-efficacy influences the choices we make, the effort we put forth, how long we 
persist (when we confront obstacles and in the face of failure), and how we feel (Bandura, 1986). In PRD theory, the 
inclusion of Bandura’s Self-efficacy principles could assist SDLs in focusing their vision and finding motivation 
toward achieving learning goals and growing in their personal relationships. 

Self-efficacy beliefs are at the core of human motivation, well-being and personal accomplishment because 
when an individual believes that his/her actions can produce the outcomes desired, s/he has the incentive to act or 
persist in the face of failure (Pajares, 2001). In 1997, Bandura contended “people’s level of motivation, affective 
states, and actions are based more on what they believe than on what is objectively true’’ (Bandura, 1997, p. 2). Since 
then, Bandura has conducted innumerable research studies to arrive at reliable conclusions. Pajares (2001) says that 
behavior can be better predicted by the beliefs an individual holds about his/her capabilities than by what s/he is 
actually capable of accomplishing. He believes self-efficacy perceptions help determine what individuals do with the 
knowledge and skills they have. 

Cognition and Understanding 

Reigeluth and Moore (1999) concluded that most theorists categorize kinds of learning in three domains: 
cognitive, affective, and physical/motor. Cognitive learning is the ability to recall or recognize knowledge, which 
results in the development of understandings and intellectual abilities/skills. In PRD theory, cognitive learning is one 
of three kinds of learning domains SDLs must master and transfer to establish, maintain and develop healthy personal 
relationships. 

Gardner (1999) suggested that the observation, assessment and improvement of applied knowledge content 
are measurements of understanding. Mental representations are validated when activated by performance. Further, 
he believes that an emphasis on performance stimulates the learner’ s acquisition of concepts and constitutes the 
best way to achieve enhanced understanding of content. PRD theory envisions learning environments in which 
practice, application and performance are conducive to learner’s understanding of differences and strengths in a 
personal relationship. 

Perkins and Unger (1999) wrote that learning with understanding yields learner’s higher engagement, 
increased active use and transfer of knowledge, and its improved retention (Reigeluth, 1999, p. 95). PRD theory 
promotes SDLs’ acquisition, performance and practice of understanding in their learning process, which could 
significantly improve the quality of lines of communication in their personal relationships. 

Learning Environments 

Corno and Randi (1999) suggest that creating an environment that is conducive to learning starts with the 
generation of appropriate instructional methods that foster self-regulated learning. This is supplemented by 
providing ample opportunities to engage in self-regulated learning and to feel successful via the identification of self- 
regulatory strategies in literature and learner’s life experience. In PRD theory, SDLs are encouraged to develop and 
nurture learning environments that foster self-directed learning for the benefit of all active participants in the 
relationship. SDLs are expected to design and customize their learning environment. 

Kovalik (1999) identifies the ‘absence of threat’ as an indispensable element in establishing healthy learning 
environment. In PRD theory, SDLs must own their learning process by having full control over their learning 
environments, actions, thought process, and conflicts. Hannafin, Land, and Oliver (1999) identify two kinds of 
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learning environments: directed and open-ended. In the latter one, the learner defines meaning, establishes learning 
needs, determines learning goals, and engages in learning activities independently. In PRD theory, SDLs define their 
learning environment depending on the goal to be achieved. 

Another important element of learning environments according to Jonassen (1999) is the “constructivist 
conceptions of learning, which assumes that knowledge is individually constructed and socially co-constructed by 
learners based on their interpretations of experiences in the world.’’ In PRD theory, SDLs engage in construction of 
new schemata by solving new dilemmas through the assistance of previous knowledge. 

Physical and Affective Domains 

Romiszowski (1999) wrote an instructional theory for the development of physical skills through which he 
identified three major methods toward developing physical skills. These are 1) imparting knowledge of what should 
be learned via expository and discovery -learning techniques; 2) developing basic skills via demonstration and 
controlled practice; and 3) developing proficiency (automatization, generalization) via shared knowledge, 
opportunities for practice, offering of feedback, promotion of transfer, and development of inner-self. In PRD theory, 
SDLs are encouraged to explore the development of physical/motor skills to improve the communication in their 
personal relationships by incorporating the above-mentioned methods. 

Martin and Reigeluth (1999) consider that affective domain refers to the development of internal changes or 
processes, while affective education refers to education for personal-social development of feelings, emotions, 
morals and ethics. In PRD theory, SDLs must develop, practice and master affective skills in their personal 
relationships to maintain a healthy balance between actions and feelings. 

Affective education includes work conducted by Goleman (1995). He researched about theories on the 
architecture of the brain and human evolution, which have shaped his understanding of how the human brain 
responds to dilemmas. Goleman says there are two systems in the brain, an emotional brain and a thinking brain. 
While the emotional brain tends to engage first when an individual is faced with a challenge, the thinking brain is still 
processing the information. Goleman identified this process as ‘emotional intelligence.’ He says that the development 
and strengthening of emotional intelligence can assist learners grapple with problems by managing their feelings, 
becoming more self-aware and improving their cognitive skills as they become more empathic (Reigeluth, 1999, p. 

488). In PRD theory, SDLs could master the affective learning domain in their personal relationships by developing 
emotional intelligence. 

Instructional Design 

PRD theory intends to provide SDLs with methods that are relevant to real-life situations as they journey to 
establish, develop and nurture healthy personal relationships. Methods could be implemented sequentially or 
cyclically. By this I mean, the methods in my instructional theory have been designed to foster growth in specific 
learning domains. Some methods are geared toward improving affective learning, others are better suited to assist 
cognitive learning, and yet others are specifically desgned to develop physical/motor learning. It is the responsibility 
and choice of SDLs to identify what methods are appropriate for the successful accomplishment of specific goals. 

Personal relationships are ever evolving and intricately complex. In many instances, after defining specific 
goals and objectives, SDLs could find that in the process of achieving those goals, new goals and objectives could 
surface. SDLs are encouraged and, to some extent, expected to re-evaluate and assess growth opportunities as they 
present, while maintaining a balance between motivation and success. 

Organization of Design 

Each of the six methods described in this section contains two or three sub-methods. To assist SDLs in 
acquiring proposed learning concepts, each sub-method is broken down into two different sections. First is Purpose, 
which briefly describes the importance and focus of each particular sub-method within the context of its method. 
Second is Suggested Application, which explores possible applications of each sub-method via a heuristic approach. 
[For the purpose of this Proceedings document, I have decided to exclude the Suggested Application sections as 
they are in the process of being published in a book.] I deemed important to include these two sections to clearly 



172 




describe, explain, and give closure to each sub-method. I believe these sections assist a learner’s ability to acquire 
and transfer knowledge as s/he explores the PRD theory. 



Methods 



Construct the Environment - Goals 

‘Construct the Environment’ is a crucial method, purposefully placed as a starting point in any given 
instance of the learning process. SDLs are charged with the responsibility of identifying the degree of motivation and 
scope of vision necessary to drive the efforts invested in their relationship into successful outcomes. Most 
importantly, SDLs need to accommodate their personal interests to those of their partner(s) via a contract (as defined 
below) with the purpose of creating a safe environment free of threat (Reigeluth, 1999 [Kovalik]) for either party. 
Constructing the Environment has two sub-methods that address specific arenas of development. 

1) Explore and create a contract of performance 

The purpose of this sub-method is to assist SDLs in naming boundaries, identifying goal(s), initiating 
engagement, and committing to actively participate in the relationship. SDLs are responsible for determining expected 
performance during the exploration of new ways to communicate with their partner(s). This is a key sub-method to 
clarify and understand roles in the relationship by giving and receiving feedback that encourages and expects active 
participation. The goal of this sub-method is for SDLs to reach a performance agreement with their partner(s) in their 
quest for growth and improved communication. 

2) Consider investment 

In this sub-method, SDLs are responsible for estimating their level of investment and intrinsic motivation 
during the learning process. SDLs must decide whether to believe their actions could produce the desired outcomes, 
and whether they have the motivation to persist in the face of challenge (Bandura, 1997). Appraising levels of 
investment and motivation could determine the likelihood of successfully reaching desired performance objectives in 
the relationship, or failing in the attempt to improve the performance of all involved parties. 

Elicit Exploration - Means 

‘Elicit Exploration’ is the next method to ‘Construct the Environment.’ After identifying boundaries and 
investment, SDLs have to search for sources of support in their journey toward improving their personal relationship. 
The purpose of this method is to assist SDLs with the identification of tools and means that could facilitate their 
learning process. There are two sub-methods to eliciting exploration. 

1) Implement new research modes 

The purpose of this sub-method is for SDLs to explore and identify tools that could assist them in learning 
more about themselves and their partner(s) (i.e. MBTI -Myers -Briggs Type Indicator). Identifying priorities and 
individual characteristics is of equal importance because these shape the focus and relevance of new learning goals 
in the relationship. Implementing strategic research modes presumes that involved parties in the relationship are 
actively interested in learning more about themselves and the other(s). 

2) Provide safe space for new experience 

The purpose of this sub-method is to create ample opportunities to experiment individual and shared 
performance in the relationship. By promoting and pursuing safe forums for communication and performance, SDLs 
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could develop healthy lines of communication based on trust, safety, and ‘absence of threat’ (Reigeluth, 1999 
[Kovalik]). Each person in the relationship is in control of the learning process, and able to form new partnerships. 

Apply Understanding - Cognitive Learning © 

The ‘Apply Understanding’ method refers to the cognitive learning domain. The purpose is to develop 
reflection skills and form collaboration partnerships as SDLs transfer and automatize performance skills. This method 
includes three sub-methods. 

1) Develop reflection skills 

The purpose of this sub-method is for SDLs to conduct self-assessments specifically for the purpose of 
identifying whether it is necessary to ask for help (e.g. counseling). Counseling is suggested as a tool for reflection 
because the process of reflecting on inner-needs and characteristics can be confusing, specially when simultaneous 
to attempting to care for and reflect on one’s partner(s)’ needs and characteristics. SDLs are expected to identify the 
need and appropriateness of asking for help. 

2) Create collaboration partnerships 

The purpose of this sub-method is to seek support from experts, mentors, literature and/or friends. This 
sub-method is designed as a support tool to the previous sub-method. Given that counseling is a viable option in 
the relationship, support tools will be needed to strengthen efforts toward developing a healthy personal 
relationship. Relying on others to assist the learning process is a tool SDLs must use time and again. This sub- 
method should be implemented as a structured support system. 

3) Transfer and automatize performance skills 

The purpose of this sub-method is for SDLs to develop abilities toward the application of learning in new 
situations. In new learning situations new knowledge can become overwhelming if it is not practiced, causing the 
understanding of content to be lost. When SDLs acquire new knowledge, they should reflect on its impact and 
consequences on themselves and their partner(s), to finally apply it to new situations. Transferring new knowledge 
into new situations will equip SDLs with tools to master different cognitive learning arenas. By transferring new 
knowledge and practicing it consistently, automatization of new skills becomes inevitable. 

Identify Alternatives - Affective Learning @ 

The ‘Identify Alternatives’ method refers to the affective learning domain. In this method, SDLs recognize 
affective limitations and needs, and identify possible solutions. This is an important method because it addresses 
the emotional state of individuals and their relationships. SDLs must be diligent and systemic in their approach to 
learning in the affective domain in order to address their own and their partner(s)’ needs respectfully and punctually. 
There are two sub-methods to identifying alternatives. 

1) Recognize needs, wants and compromises 

The purpose of this sub-method is for SDLs to identify individual and relational orientations and 
collaboration patterns. SDLs are in charge of dis tinguishing and sharing what they would like to have or see happen 
in their relationship, what they need to have or see happen in their relationship, and what they could give up or 
compromise in their relationship with the ultimate goal of learning and growing. Identifying needs, wants and 
compromises is a conflictive process for most individuals and relationships. It requires SDLs to be timely, purposeful 
and determined in their attempts to learn from themselves and their partner(s). 

2) Implement possible alternatives 
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The purpose of this sub-method is for SDLs to actively identify alternative solutions and 

best ways to implement them. SDLs must purposefully engage in meaningful dialogue to suggest, model 
and understand different approaches to addressing needs, wants and compromises in the relationship. 

Assist Practice - Physical/Motor Learning @ 

The ‘Assist Practice’ method refers to the physical/motor learning domain. SDLs are encouraged to address 
physical/motor needs periodically in their personal relationships because these usually account for a large 
percentage of the needs and wants of each party in the relationship. This method focuses on practicing new 
performance skills, enabling feedback opportunities, and appraising motivation. There are three sub-methods to 
assisting practice. 

1) Practice new performance skills 

The purpose of this sub-method is to provide ample opportunity to apply new skills. The learning and 
development of physical/motor skills is easily neglected in personal relationships. Allowing safe spaces for practice 
could ensure improved communication and enhanced participation in the relationship . 

2) Enable feedback opportunities 

The purpose of this sub-method is for SDLs to find ample opportunities to evaluate success or interpret and 
analyze failure. As new skills are applied and practiced, success and failure performance areas should be determined 
and addressed. SDLs are encouraged to embrace performance results and carefully analyze patterns of behavior and 
response to needs and wants. 

3) Appraise motivation 

The purpose of this sub-method is for SDLs to re-assess intrinsic and extrinsic motivators. When physical 
learning is being practiced for mastery and automaticity, assessment of motivation levels is a key element that could 
elicit success or failure in the performance. Evaluating levels of motivation could determine long-lasting efforts of 
improvement. 

Facilitate Evaluation - Closure 

‘Eacilitate Evaluation’ is the last of the sequential methods offered by the PRD theory. The purpose of this 
method is to give closure to a stage, to assess areas of growth, and to discover next steps. SDLs are expected to 
actively implement this method before moving to the identification of new goals and objectives. There are three sub- 
methods to facilitating evaluation. 

1) Give closure to a stage 

The purpose of this sub-method is for SDLs to develop the ability to recognize the end of a growth, comfort, 
or faulty stage/situation in the relationship. A growth stage is one that has been determined successful by involved 
parties in the relationship. A comfort stage is one that impairs SDLs to move forward and begin addressing new 
challenges. A faulty stage is one that has been determined a failure by any involved parties in the relationship. 

SDLs are urged to acknowledge and address the existence of any of these three stages in their relationship. SDLs are 
responsible for identifying the need for closure to any given stage. 

2) Assess areas of growth 

The purpose of this sub-method is for SDLs to create environments and practice communication routines 
that elicit feedback about accomplishments. SDLs should focus their strength, effort, motivation and decision- 
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making on the accomplishments of their learning process. Growing stronger and healthier relationships is very hard 
work. Acknowledging accomplishments is a reward that needs to be systematically implemented by SDLs. 

3) Discover next steps 

The purpose of this sub-method is for SDLs to identify new challenges and establish a vision for next steps 
in their relationship. This sub-method is conducive to a new ‘constructing the environment’ method, which is 
designed to begin any learning effort in the relationship. SDLs are expected to design and envision next steps 
toward new learning and growth. 

Discussion 

PRD theory is meant to be but only a first step toward providing SDLs with tools to improve their personal 
relationships. It briefly addresses parallel issues of importance such as motivation, self-esteem, vision, effort, and 
investment, among others; these, and many other learning elements must be present during the decision-making 
process, construction, implementation and assessment of SDLs performance in their personal relationships. 

PRD theory offers SDLs a generalized and systemic approach to daily functioning in a personal relationship. 
The scope of my theory precludes it from being clinical or scientific in its approach or content. The purpose of the 
PRD theory is to support SDLs in developing skills that could assist them in identifying, establishing, developing 
and nurturing healthy personal relationships. 

The Presentation of the PRD theory in the AECT 2002 conference described the application of one sub- 
method over a period of six months. The purpose of the research conducted was to ground one instance of the 
theory in practice to assess its value and effectiveness. If you are interested in receiving information about this 
study and its impact in the lives of study participants, please contact me at spascoe@indiana.edu. 

Personal Investment 

I have been involved in a meaningful personal relationship for the past fifteen years. During this time, I have 
developed communication skills in this relationship that have strengthened and sustained it through time. However, 
it has only been by chance that in my experience I have found necessary skills, tools, and communication patterns for 
my relationship to survive. By this I mean, no one ever taught me how to relate or behave in a meaningful 
relationship. I have had to force myself to think in new ways, re-focus my goals and search for new meanings; this 
has meant being intrinsically motivated to investing myself when I am emotionally drained, stumbling into walls to 
change decision-making patterns, and picking-up the pieces to continue making it worthwhile. 

Through this pragmatic process I have wondered, why is it that I was never provided with tools to manage a 
personal relationship while growing up. As I started turning to friends for support, advice, mentorship or modeling, I 
could not find healthy relationships from which to feed my desire to improve, grow and learn. I realized that most 
adult meaningful, long-lasting relationships struggle to make sense of their nature and purpose. Therefore, I saw an 
imminent need to design an instructional theory (PRD) addressing this aspect of human performance. 

It is my intent to propose a theory that systemically and systematically provides SDLs with tools and 
structures that support their efforts as they struggle and journey toward developing healthy patterns of 
communication in their personal relationships. I believe meaningful relationships in the life of an individual could 
become the axis of all personal and social functions. They could serve as a support system in times of pain and 
dismay. They could provide strength in times of success and growth. But most importantly, healthy meaningful 
personal relationships could serve as the backbone to a happy and balanced life. I wonder why I have not yet found 
a publicly accessible instructional theory that addresses this aspect of our human performance. 
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The PRD theory must recognize that personal relationships are ever evolving and that new patterns of 
communication and new schemata must be constructed on a daily basis to sustain growth efforts. Therefore, PRD 
theory must envision the inclusion of new development steps toward supporting its goal and methods. My goal in 
publishing the PRD theory as a means of publicly stating: we need instructional theories in this area of human 
performance. 
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Global Snapshots of E-Leaming: A Series of Case Studies 

Ke Zhang 

Jonathan U. Dougherty 
Rakel-Kavena Shalyefu 
Tugay Tutkun 
Abstract 

This paper will report a series of case studies of e-learning, each of which represents a different region of the 
world to get a global perspective. An example case from each region of the United Nation's division of the world, was 
studied, described and compared to the others. These regions included: Africa (Sub-Saharan), Asia and the Pacific, 
Europe, North America, and Latin America and the Caribbean. The only region not yet represented in our case 
studies was the Arab States (North Africa and the Middle East). 

Introduction 

This paper includes a series of case studies on e-learning in varied regions of the world. In this paper we 
first briefly discuss e-learning perceptions and practices in general. Then five cases of e-learning in a different region 
of the world will be presented. An example case fromeach region of the United Nation's division of the world, will be 
studied, described and compared to the others. These regions include: Africa (Sub-Saharan), Asia and the Pacific, 
Europe. North America, Latin America and the Caribbean. 

These cases represent not only various geographical regions of the world, but also various types of e- 
learning applications, including e-learning service firm, higher education institutions, and industry using e-learning 
for training and knowledge management. 

Each case will start with a description of the region, focusing on its culture, technology status, and value of 
learning. Then the authors will zoom in and focus on the selected case with detailed discussions of its e-leaming 
strategy, its propensity for sustaining e-leaming and its vision. As Rosenberg (1999, 2001) recommends, the authors 
will analyze each case with the following criteria, how e-learning is viewed and aligned with business, the current 
state of e-learning and technology, funding, evaluation, coordination, competition, and instructional development 
capabilities. In each case the authors will make recommendations with dos and don’t for successful e-leaming 
application. 

Zhang will report a case study about an e-learning service company in China, focusing on the cultural 
factors that impact the design and implementation of e-leaming. Shalyefu will discuss the challenges faced in 
implementing e-learning solutions in Namibia. Dougherty will look at the e-leaming implementation in a building 
company in the United States. Tutkun will report the e-learning status in a largest Turkey university. 
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Global Snapshots of E-Leaming: A Series of Case Studies An E-Leaming Dot 

Com in China 

Ke Zhang 
Abstract 

This paper reports a case study on an E-Learning service company in China, which will be referred to as EL.com 
throughout the paper. This paper starts with a brief review of the current e-learning status in China, and follows 
with a selected case of e-learning service company. The case study provides detailed description of and analysis on 
the perception and practice of e-learning, and concludes with recommendations and discussions on the key 
elements for engaging e-learning, as indicated in this particular case. 

The Bigger Picture: E-Leaming in China 

Distance education has progressed in China aggressively since the 60s with nationwide television systems 
(Carr-chellman and Zhang, 2000; Gao, 1991; Howells, 1984; Keegan, 1994). Varied technologies, such as telephone, 
television, satellite televising systems, correspondence, and face-to-face learning centers across the country have 
been integrated for distance education since then, which is typically vocation-oriented (Carr-Chellman and Zhang, 
2000 ). 

In recent years there has been a surge of online learning in China (Carr-Chellman & Zhang, 2000; CERN, 
2001). In 1999, the central government of China and the Ministry of Education announced a development plan for 
“modern open distance education” with networking technologies. Since then, the government has taken a series of 
initiatives to promote joint efforts from varied parties for developing “modern distance higher education” in China. 

As of 2001, two years later, China has thirty-one accredited universities providing on-line education packages and 
has approximate 190,000 registrants (CERN, 2001). At the same time, increasing number of non-traditional learning 
institutions become available, most of which provide web-based certificate programs. 

In the past three years, internet and networking technologies have been utilized in all level of educational 
institutions, from K-12 to higher education, for varied purposes (e.g., information and communications, teaching and 
instruction, and other learning activities). Many institutions are now providing some sort of e-learning opportunity to 
enrolled students and/or public. Most of the applications, however, are limited to communication and information 
updates. A few corporations, with partnership with higher education institutions, are in the early stage of active 
marketing for the new concept of e-leaming. 

An E- Learning Dot Com: General Information 

Established in 1999, EL.com is one of the few companies in China providing e-learning products and 
services to individuals. Although named as a vendor for general e-learning, the company specializes in information 
technology and other e-commerce related learning with an e-commerce certificate program. As the targeted customers 
are individuals and not organizations, the current e-learning services focus on individuals, and for organizations the 
services is limited to a lower rate for company /group registration. The company offers some highly demanded 
learning packages free to public, such as web design and authorware tutorials. Currently eleven instructors are 
available for twenty-six courses and five course packages. As of fall 2002,it reports approximately ten thousands 
online learners daily, mostly from China and some from UK and Canada. 

The brick and mortar office of EL.com is in Beijing, capital and cultural center of China. The geographic 
location makes it more convenient to recruit instructors and subject matter experts among faculty members from the 
best universities of China. As Chinese culture highly values academia and university brand names have particularly 
strong marketing power (Carr-Chellman & Zhang, 2000), this partnership has helped the company establish credibility 
as a learning institution, and thus helped building more confidence in the learning programs. 

The course price ranges from 30 Ren Min Bi Yuan (less then 4 US dollars) to 250 Ren Min Bi Yuan ( 
approximately 30 US dollars) per course and 1,500-3,100 Ren Min Bi Yuan (approximately 180 - 380US dollars) per 
package. 
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Strategic Partnership 

EL.com has established partnerships with the Chinese Ministry of Information Industry (CMII) and 
Tsinghua University (THU). As a result of the partnership with CMII, learner who have successfully completed the 
e-commerce learning through EL.com are granted CMII-issued e-commerce certificate. The e-commerce certificate 
program has thus attracted a lot of interests and participation, and helps EL.com establish a unique, strategic position 
in competition against rivals. The partnership with THU eases the efforts to recruit instructors and subject matter 
experts from the high status university, and the big names further attracts learners and strengthens the company’s 
credibility as a learning institution. In the partnerships EL.com is responsible for design and development of the e- 
leaming system in addition to delivery, management and marketing of it. 

The strong partnerships with CMII and THU have enabled EL.com to strategically position itself as a 
provider of convenient learning with famous universities for nationally accredited certifications in addition to all the 
general advantages of e-learning. 

Considering the current technology status in China (see Carr-Chellman & Zhang, 2000 for details), EL.com 
has built partnership with some local learning centers, who are responsible for local recruitment and management. 
Thus el.com is able to provide e-leaming services to meet the high demands for e-learning, even in less-developed 
areas. 



Status Quo 

The company’s focus, as of November 2002, is on individuals, not organizations. It is clear that EL.com 
views e-learning as technology-leveraged learning with a series flexible learning modules. 

Consistent with China’s current technology status, e-learning through EL.com does not require heavy 
technology, plug-ins or broadband. The only technology requirement is a standard computer and Internet access. 

The e-leaming system, including the courseware, information dashboard, instructor’s course management system, 
and learner’s learning management system, is available through the world wide web. All the learning activities 
happen on the Internet or, if chosen by learners, in local learning centers. Through the geographically dispersed 
learners and the World Wide Web, EL.com has successfully spread the concept of e-leaming. 

EorEL.com, e-learning is the business. Eunding for the company comes from private parties, commercial 
banks, and tuition revenue. Currently the company evaluates e-learning based on learning outcome and learners’ 
satisfaction. Operating as business, learners are considered customers and clients, and learner satisfaction is critical 
for the company’s development. Thus learners’ evaluation, comments, suggestion and other feedback are highly 
valued as the evaluation of the “consumed” e-learning. Those evaluations, if published through the online forums, 
are open and available to general public. The open evaluation data is very telling and attracting to general public. It 
helps educate general public about e-learning, which in turn, promote e-learning in general and may develop business 
opportunities for EL.com. 

El.com’ s major rivals are similar e-leaming service companies in China and higher education institutions 
offering web-based course and training programs. Yet because of the unique focus on e-learning and CMII-issued 
certificate, no rivals are competing for the exactly same market with EL.com, and it is not likely to be any in the very 
near future. Thus EL.com has successfully established a strategic position in the market. 

As a start up company, EL.com has only about ten full-time IT professionals and staff in addition to the 
instructors and SME. 



Vision of E-leaming 

Currently the company has limited its e-learning services to individuals and has not considered 
organizations as potential clients. Such strategic positioning is consistent with their view of e-learning. As indicated 
in their current e-leaming model, EL.com views e-learning as a new learning method beneficial to individuals with 
flexibility and convenience, yet its potential to benefit an organization with features as knowledge management and 
task enabler is not of an interest to EL.com. 
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E-learning is a brand new idea in China. As one of the pioneers, EL.com has little to learn from others but 
has to learn while practicing. With general understanding that learning needs to be flexible to meet varied needs of 
different learners, EL.com offers portable learning modules and enables learners to monitor and manage their own 
learning through the web-based learning management system. 

According to its statement on the company website, its mission is to promote e-commerce by providing 
related e-learning. Thus all the activities are devoted to e-commerce and related knowledge and skills. 

learning Methods 

EL.com markets e-learning as web-based, convenient, flexible and self-adjustable learning. The EL.com 
website serves as the major venue to the e-learning services it provides. The website is designed with the following 
four categories, course and service information, cyber space for e-commerce, learner center, and learning fomm. The 
first two categories are open to general public, free of charge, and the last two, learner center and learning forum are 
available only to registered learners. Course and service session provides detailed information about the courses and 
learning methods. Cyber space for e-commerce serves as an information dashboard with e-commerce related 
information, discussion and communication. It has actually emerged as an online community of e-commerce 
practitioners and others who are interested in e-commerce. Learner center is a learning management system for 
registered students, where they can monitor the progress, conduct evaluation and adjust the learning pace. 

It offers all the courses in portable modules, and students can choose to skip some of the modules and 
adjust the leaning pace accordingly. All the course materials are offered through email or downloadable from the 
internet. Learners also have access to asynchronous online forums to communicate with the instructor and peer 
learners. Lor some certificate programs, learners can choose to enroll in a face-to-face training session instead of 
online learning to fulfill the certificate requirement. 

The company’s website, which includes information dashboard, courseware, learning management system 
and virtual community, illustrates that EL.com is trying to offer e-leaming to the public through means of information 
sharing, community support, and interactive collaboration together with learning and instruction. With easy and 
open access to information and community support, EL.com promotes accidental learning, knowledge sharing and 
collaboration together with formal learning and instruction. 

Instructional Design Issue 

In order to gain first-hand experience of e-learning services provided by this company, the author has 
registered as a student and taken two courses from El.com, both of which were free of charge for public. 

As a registered student, the researcher is able to experience e-learning as they define. In general, the 
learning experience is easy and rewarding. As an experienced online learner, the author dose not find it very difficult 
to complete the web-based learning. The learning has three major components, instruction, case study, and peer 
collaboration. The real-world cases are very interesting and challenging, and so are the peer collaborations. However, 
the instmctional materials appear like shovelware (Rosenberg, 2001) at the first sight. In another word, they look like 
duplication of classroom lectures. The text reads like word-by-word transcript from a classroom lecture. And in most 
cases, they are simply e-reading. But on the other hand, reading the text is very similar as listening to a lecture in 
person. The text is written in a personal, dialogue style in most cases. Thus it reads more personal and approachable, 
and the author, as a learner, feels easily attached to the materials. As a Chinese, the author understands that many 
Chinese like to learn with the masters, especially in person. So for learners like personal instmction, this type of 
“shovelware” of a lecture from a famous scholar is not a bad thing at all. Many courses use real-world business cases 
to illustrate the points, and the major assignments are all real world projects, which are very exciting and motivating, 
as indicated in students’ evaluations. So in general, the course materials and learning activities are good. 

There is no real time instruction or interaction with the instructors. Interactions are only available through 
the learner’s forums or personal emails. With the time differences, my personal experiences with those 
communications are rarely in a timely manner, but I suspect it would be much less of an issue for domestic learners 
residing in China. However, if the company is interested in the overseas Chinese market, as it claims, they would need 
to address the time issue carefully to achieve customer satisfaction. 

Discussion 
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El.com has a good understanding of the dominant culture about learning and the current technology status 
in China as well. Accordingly, it has tailored e-learning with limited technology requirement and established strategic 
partnerships with the influential government and well-known universities. With a simple stated, straightforward 
mission, EL.com is able to stay focused on the targeted market and population. 

With the click-and -mortar operation model, minimum technology requirement, and free or low-cost learning, 
el.com is able to provide very affordable and accessible e-learning services to the targeted population. 

Its partnership with the government and a best university immediately helps EL.com build credibility as a 
leading company in e-commerce related e-learning. The partnership with the government also helps EL.com build an 
image as an e-learning champion with powerful authority. 

With the online community emerged from the el.com website, the company has successfully attracted not 
only registered learners but also general public, and potentially people from the cyber community will take some of 
the e-commerce courses with EL.com and increase its reputation as a leading learning community for that subject 
area. In addition, for enrolled learners, this cyber space serves as a supporting community, which further promotes e- 
learning. 

Erom the online communications that the author has accessed to, it is clear that the online community has 
established a very positive learning culture highlighted with optimism, energy, and hardworking spirit. The free 
courses from EL.com have attracted more people to participate in e-learning, and thus further promote the learning 
culture. Such culture in turn eases the efforts to overcome difficulties and challenges associated with e-learning as an 
innovation. The company website does not provide successful stories to promote e-learning, yet the real-time 
updated students’ evaluation vividly illustrates the success of EL.com. It is obvious that learners are very excited 
about the new learning method as well as e-commerce, the subject that they are all interested in and enthusiastic 
about. 

Through the online communities, the general e-commerce community and the e-learning community, the 
company helps learners and general public understand and implement e-learning through communications with a 
diverse population. I am very surprised to find few complaints about the new learning method, and think the 
company has done a great job with the communities’ help. 

Personally, I am very happy and excited with the e-learning experiences with el.com. And I believe that the e- 
learning they provide has met people’s needs with all the technology restraints. As an instructional designer, 
however, I would like to see more interactive collaborations among peer learners and between learners and the 
instructor. Being very aware of the lack of instructional design professionals in China, I would like to suggest that e- 
learning companies in China seek partnership with instructional design professionals or partially outsourcing 
instructional design tasks. 

Although for such a small business it is understandable to not have an e-leaming system, I would still like 
to recommend a knowledge management s ystem for the learners and the company itself. A knowledge management 
system will enable learners to better manage the often over-loaded information and promote knowledge sharing and 
collaboration. Eor the company itself, if the instructors from the top universities and the government can have a 
shared knowledge management system for knowledge sharing and collaboration, it will further boost their motivation 
for e-learning. To achieve that, the company needs to secure funding from various channels, including income stream 
from their e-learning services. 
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Global Snapshots of E-Leaming: A Series of Case Studies An E-Leaming 
Application in the United States’ Building Industry 

Jonathan U. Dougherty 
Introduction 

The following abstract is based on a case study of the e-learning strategies developed by a nationally 
recognized construction firm in the United States. The firm concentrates on construction projects in industrial, 
institutional, and commercial sectors and also serves clients in the advanced technologies market. At the request of 
the company, any identifying information about the firm will be withheld. To meet the learning needs of the firm’s 
employees, the company established a corporate university. 

The firm offers an appropriate technology infrastructure to the majority of its employees. The issue of 
connectivity at individual construction project sites must be addressed. Time and money are the key factors in 
providing a solution to this infrastructure concern. As new technology is developed, it will be easier and cheaper to 
implement an effective infrastructure on an enterprise-wide basis. 

More important than issues of appropriate infrastructure is company culture and the readiness and 
willingness for all of the company’s employees to take advantage of such technologies. This culture does not merely 
exist at this firm but resonates throughout the entire building industry. Construction professionals can be 
considered traditionalists - they are used to drawing by hand not with a computer, they like to be on the 
construction site to see and feel what is going on, they like to learn in a classroom with a recognized expert. Clearly, 
this is a culture, which will not readily embrace e-learning. The adverse understanding of technology is indeed a 
problem; however, this company has already committed so much into issues of infrastructure, software development 
for project management, and corporate learning that letting this get in their way would be disheartening. The issue of 
company culture must be dealt with and overcome. 

On the opposite end of the spectrum is the corporate university’s ability to build support from top 
management. Not only are company executive “buying into’’ the corporate university, they are singing its praises in 
newsletter articles and developing online classes, which they will teach. Part of the university’s success is 
stakeholder management. All too often, the word stakeholders is accompanied with a negative connotation of people 
or groups who are quick to point out weaknesses but shy away from offering solutions. This is not the case in this 
instance. All stakeholders, including senior management, play an integral role in the operation of the corporate 
university. This is definitely one of the reasons for the success of the firm’s corporate university. 

E-Leaming Strategy 

For reasons of confidentiality, the firm was reluctant to divulge specific information on the development and 
preparation of their learning strategy; however, detailed information was received on the corporate university’s 
stakeholders and their purpose. The firm has adopted the best qualities of stakeholder management into their 
corporate university. To effectively incorporate the ideologies of the various stakeholders, this corporate university 
has provided these people with the ability to actively participate in the development, administration, and everyday 
activities of the learning process. 

Propensity for Sustaining E-Leaming 

This company believes that offering employees an e-learning environment allows them the ability to take 
charge of their careers by exploring and defining their career aspirations and participating in the firm’ s success as a 
leader in the construction industry and in the area of providing lifelong learning opportunities to employees. 

The corporate university has a realistic perception of e-learning and is working to build these educational 
initiatives into a premier construction education and career development source for its employees. The use of a 
learning management system and an educational intranet website (the company’s version of a knowledge 
management system) is effective to date and addresses the current needs of its users. As e-learning is more 
appropriately integrated into the learning vision, these systems should be updated to mesh with employee usage and 
business strategy requirements. 
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Vision 



While the firm’s primary focus is the construction business, the company’s vision statement revolves 
around people, both clients and employees, and the desire to provide opportunities for continuous learning. This 
vision, while seemingly general, is quite telling in its simplicity. The vision statement uses powerful language and 
allows the corporate university the opportunity to develop and utilize unique and cutting-edge technologies in their 
pursuit of providing an environment indicative of knowledge sharing and continuous learning for all employees. The 
firm’s mission and vision statements and the adoption of e-learning technologies position the company to not only 
enjoy a productive future in providing educational initiatives to employees but also to be a leader in construction 
education throughout the United States. 

View of E- Learning / Aligning with Business Objectives 

The company has a well established corporate university which serves as the cornerstone of education and 
training and incorporates applications of electronic learning. To better articulate the value of a corporate university, 
the firm has done an excellent job of providing senior management and employees with a knowledge base on the 
importance of continuous learning. This corporate university also recognizes and embraces the ideologies of e- 
learning, and defines e-learning as online learning with occasional utilization of CD-ROM technologies. The primary 
emphasis of employee training is technical skills, i.e. courses related to construction management and project 
controls. They do, however, identify the need for computer and Internet skills and an equal need for leadership and 
interpersonal education. 

Aligning business strategies with the mission of the corporate university is a cornerstone to its success. 
Business principles must be appropriately integrated into an e-learning strategy to continue to strengthen the role of 
information and communication technologies. By implementing these core values into the corporate university, the 
firm can offer a fusion of business strategies and corporate education. This unification will be especially beneficial in 
rationalizing e-learning and all learning initiatives during difficult economic times when employee training usually 
takes a backseat to the everyday business of the company. 

Current State of E-Learning and Technology 

Based on the previously mentioned vision statement for the firm, it is easy to see their continuing dedication 
to their clients. In addition, the firm places much effort on the attraction and retention of highly motivated employees 
who can accept the challenges of working in the constantly changing building industry. The reasoning for the 
building industry being described as changing is due to the wide variety and complexity of construction projects an 
individual can be a part of during his/her career. 

The corporate university is integral to the firm’s passion for continuous learning and challenges all 
employees to learn while providing the knowledge and skills necessary to excel both within the firm and the 
construction industry. The corporate university strives to balance employee growth and advancement through job 
experience and company training. The university facilitates access to learning through face-to-face instruction, 
online courses, a learning management system, an education intranet site, and dedicated learning professionals. 

The beacon of the corporate university’s e-learning initiatives is the recently introduced learning 
management system (LMS). This learning management system is made possible through a partnership with a well- 
known LMS vendor and provides employees with: 

• Course catalogs outlining face-to-face and online courses, videos, books, and CD-ROMs 

• Personalized employee transcripts to track learning history 

• Up-to-date announcements 

• Self-registration 

• Automated management approval processes 

• Track and renew certifications (e.g. Occupational Safety and Health Administration) 

With regards to the company’s current state of technology, the firm has been recognized by the building 
industry as a leader in the area of cutting-edge information and communication technologies. However, it is 
important to point out that the company does experience connectivity issues due to the vast geographic regions. 
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which it serves, and the issues invoived with providing Internet and intranet service to individuai job sites 
throughout the United States. Despite the overaii readiness of the firm to provide IT services to aii of its empioyees, 
there are empioyee readiness factors, which must be considered. Some empioyees gravitate to the use of Internet 
and intranet appiications for managing construction projects and participating in oniine iearning, and some do not. 
Issues of company culture and its reiation to information technoiogy are omnipresent. 

Funding 

Whiie specific funding information was withheid by the firm. An articie in a company-wide newsietter 
addresses issues of funding by stating that the firm’s annuai training expenditures exceed $i000 per empioyee. This 
staggering doiiar vaiue truiy iiiustrates the firm’s dedication to aii aspects of iearning, inciuding e-iearning. The 
corporate university is provided with no direct cost to the empioyee. 

Evaluation 

For reasons of confidentiaiity, the company did not disciose the methods of evaiuation for corporate 
iearning activities. The reviewer estabiished an action pian for the corporate university, and it is presented here to 
address the evaiuation process. It was important for an e-iearning strategy to address criticai success criterion, and 
that these factors be a driving force in the impiementation of effective and efficient e-iearning initiatives. The 
foiiowing criterion shouid be addressed: 

• Abiiity to meet iearner expectations 

• Abiiity to attract and retain empioyees 

• Abiiity to transiate iessons iearned from courses to buiiding projects 

• Learner satisfaction with e-ieaming experiences 

• Abiiity to provide meaningfui e-iearning experiences for aii empioyees at Internet speed 

• Ability to offer accurate, relevant, and complete information using e-Ieaming 

• Maintaining cost effective solutions 

• Adhering to budgetary constraints as mandated by senior management 

• Justifying costs/expenditures to senior management 

Much emphasis is placed on continuous learning, and undoubtedly, the firm is a pioneer in providing 
structured employee education; however, now the focus of the corporate university must shift to the continual 
enhancement of information and communication technologies into each employee’s career development curriculum. 

It is the opinion of this critic that these objectives can be successfully improved by continuing to implement 
a comprehensive learning architecture, which addresses ideas of face-to-face instruction, e-learning, and on-the-job 
training. 

Coordination 

Another important issue the corporate university has faced and will continue to face is outsourcing and 
vendor management. When the learning management system was in its infancy, the corporate university team 
researched available systems and developed a checklist of needs to help in the selection of a LMS provider. As the 
university continues to grow and improve on their e-learning strategy, more issues of outsourcing and management 
will become evident and steps should begin now to deal with such items. In essence, the key element is vision. The 
team must deal with the daily operation of the university; however, without appropriate vision and the ability to act 
on that vision, the future success of any corporate university will be diminished. 

Competition 

This corporate university is truly on the leading edge of the successful integration of educational initiatives 
with employee’s career development. Being a pioneer means that for the most part there were no mentors for the firm 
to look to in this endeavor; however, the corporate university now has the opportunity to be a mentor for a nearly 
$600 billion construction industry. This is an awesome responsibility to undertake, especially given that the 
company wants to maintain the highest of quality within the organization and not provide competitors with “trade 
secrets’’ from the substantial work that has been accomplished. The firm must prepare and take the necessary steps 
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to act on a plan to share their considerable knowledge in providing corporate education without compromising the 
integrity of their program. 



Instmctional Development Capabilities 

It is important to recognize the employees, the clients, and the corporate university team. The team consists 
of a meager staff of three, which includes a director, an instructional designer, and a learning technologis t. The staff 
serves over 1000 employees of the company and in turn serves all of the firm’s clients. Clearly, issues of staffing and 
instructional design capabilities must be more appropriately aligned with the number of learners. 

Recommendations 

The following are recommendations based on the case study: 

• Work to continue strong commitment from upper management and all stakeholders 

• Better utilize employees as Subject Matter Experts (SMEs) in the development of online courses 

• Continue internal and external communication and visibility of company’s commitment to life-long 
learning 

• Build course offerings to include all employees and provide more and better instruction in 
leadership and interpersonal skills 

• Provide appropriate infrastructure to all employees 

• link education to other online firm resources, including project websites 

• Provide appropriate training for all instructors 

• Address issues of technophobia and company culture within the company 

• Expand corporate university staff to address the needs of a fast growing company 
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Global Snapshots of E-Leaming The Challenges of E-Leaming Solutions in 

Namibia: The Current View 

Rakel-Kavena Shalyefu 
Introduction 

The Information and Learning Resource Center Project (ILRC Project) is a capacity building of the Human 
Resource Development Project (HRDP) in Namibia with partly sponsorship of the African Development Bank (AfDB), 
the Arab Bank for Economic Development in Africa (BADEA) and the Namibian government. 

The ILRC is housed by the University of Namibia (UNAM), but it is a national project. UNAM spearhead 
the project without owning it. The primary purpose of this project is the integration of the latest technological 
resources. The ILRC is expected to facilitate the improvement of educational performance through increasing use of 
relevant information in all formats while it also functions as a national reference library, a “library without walls,” that 
will open up resources making Namibian sources available worldwide as it accesses foreign databases and other 
electronic share waves. The ILRC is a network of human and material resources, and will enable information to be 
used in formal and nonformal education. The major objective of the HRDP is to train users in Information 
Communication Technologies (ICTs) and Instructional Design (ID) through e-learning strategies. 

ILRC’s view of e- learning 

ILRC views E-Learning as: the use of technology ‘from print to electronics’ to deliver solutions, to enhance 
and support knowledge and performance. In this view, E-Leaming includes: books, audio tapes, video tapes, CD- 
ROM, Web-based Training, the Internet, CD-ROM, Computer-based Training, Web Conferencing, Web Meetings, 
the Internet, the World Wide Web (WWW) and other online performance support tools. 

E-Learning is seen as anything delivered, enabled or mediated by information communication technologies 
for various purposes such as: learning, instruction, information, knowledge and training. E-Leaming for ILRC 
therefore does not mean online learning only; it should be a comb ination of different kinds of technologies including 
print. Rosenberg (2001) supports this view by stating that E-Leaming is networked, delivered to end-user via 
computer, go beyond traditional paradigms of training, but it is not tantamount to distance learning. So we can say E- 
Learning is part of distance learning, information, training and instruction but it is a disservice to the advancement of 
E-Learning to call it ‘alternate learning’ or ‘alternate delivery.’ 

ILRC’s technology status 

Computer technology and the Internet brought excitement to the world of knowledge, but there are 
limitations. There are Regional Outreach Centers that have been furnished with Internet access and other equipment 
like portable screens, stand-alone screens, LCD Projector, video players, HP Scanners, CD ROM ReaderAVriter, etc. 
There are no web-based courses for the pioneers to see yet, but they are referred to web sites (information) to add to 
their knowledge and also to enhance their understanding of information communication technologies. 

There are a lot of Internet problems, because of various reasons: 1) an archaic, outdated network with its 
outdated equipment, 2) a limited bandwidth (128kb leased line) is a major impediment to browsing the Internet 
effectively. To upgrade these, there are financial implications involved, 3) modems with slow download speeds are 
experienced, 4) vims outbreak, though a worldwide phenomena, added a denial of service (DOS) to many popular 
websites. This virus activity on the Internet has placed an added strain on limited networking resources. 5) Certain 
sites have been blocked (which made the management team very unpopular with fellow colleagues), even if it was 
done in hope to free up some bandwidth, so that customers could effectively access popular sites (like CNN 
especially with the World Trade Center saga). 

The mission and vision of e- learning in the ILRC 
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The mission and vision of the ILRC project is well -represented by the diagram of Grabowski and Small (1991) 
which indicates e-learning, e-instruction and e-knowledge sharing shoulders under the umbrella of information with 
the supporting poles of networked human resources and networked media resources. 

The diagram below demonstrates this concept: 




Networked 

Human 



Networked Media 
Resources 



Adapted from: Grabowski, B. and Small, R. (1991). “Information, Instruction and Learning: A Hypermedia 
Perspective.’’ Performance Improvement Quarterly. 4(3), 3-12. 



The mission of ILRC is to use the powers of information communication technologies to increase access to 
learning, instruction, information and training. A combination of technologies will be provided to support outreach 
operations at different levels of the education systems. 

It is the vision of the ILRC to take information and education to the homes of the Namibians. Though not all 
users have computers in their homes, there are Regional Outreach Centers, Open and Distance Learning Centers and 
commercial points that will assist e-learning to become a supporting mechanism for learning at a distance. 

ILRC capabilities to launch and sustain e- learning 

The major assumption of the HRDP is that the University of Namibia (UNAM) would be able to establish 
staff and maintain the ILRC. Good reason to believe this stand is the fact that Namibia has only one university and a 
polytechnic to serve a national population of 1.8 million people. This diverse and widespread population demand 
learning at a distance as a solution to avoid over-crowdedness at the UNAM main campus or satellite campuses and 
centers. This demand can be met by an e-learning strategy that the ILRC offers. Now that Namibia has a local 
technical infrastructure, the first of its kind in Southern Africa, it can respond to the market demand especially by 
offering enhanced quality training, instruction and information in fields of science, engineering, agriculture, 
technology and education. The key stakeholders who will be networked through the ILRC management team can 
cater for the demand of knowledge and skills. 
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The networked resources, the Internet opportunities and multimedia sources will be able to launch and 
sustain e-learning, especially in cooperation with Namibia’s state of the art fiber optics telecommunication. 

Instmctional development capabilities 

Currently there are few local people with information commu nication technologies expertise, but various 
stakeholders have been asked to select capable candidates from their institutions to be trained in information and 
communication technologies and in instructional design over a period of time and enable them to plough back their 
skills at the community level. ILRC regards this as a cascading model. 

The role of Instructional Designers in the ILRC at the University of Namibia will be to: 

• Identify and communicate best practices for applying teaching and learning theories and instructional 
technologies to curriculum design. 

• Evaluate diverse individual and department training needs in the area of instructional technology. 

• Assist lecturers in the design and development of engaging, interactive and instructionally sound 
materials that address various learning styles and are accessible to all students, including those with 
disabilities. 

• Design and deliver training programs, workshops and seminars on Internet, multimedia and other 
instmctional technologies and their implementation. 

• Plan and coordinate training programs with ICT training staff. 

• Research, write and produce clear documentation and other instructional materials (websites, 
multimedia and online tutorials) to support instructors in the performance improvement process. 

• Keep instructors informed about instructional-technology events and resources. 

• Organize instructional-technology activities and events for personnel staff development. 

• Serve on the professional staff development teams of the various stakeholders and provide leadership 
in instructional technology planning, development, implementation and evaluation. 

• Remain current on emerging technologies that have potential value for teaching and learning, through 
evaluation of training sessions, conferences and publications. 

The Instructional Design Unit will be responsible for the design and development of high quality materials 
that meet learners’ needs. It will also provide training sessions on instructional design on a competitive fee paying 
basis (HRDP Report, 2001); run workshops on how to improve teaching and learning, and conducting e-learning with 
partnership with the subject matter experts (SMEs). 

Coordination of e- learning 

According to Horton (2001), moving into an e-learning venture is a complex, sprawling, and dynamic 
endeavor. It requires a range of skills, talents, knowledge, and viewpoints. The best way to meet ILRC requirements is 
to assemble a multitalented, dynamic team. Some individuals can perform multiple roles and not everyone is needed 
on every phase of the effort. At every milestone, there will be need to adjust team membership to better accomplish 
the next phase of work. The composition of the team will depend on several factors, such as 

• Size and scope of the project 

• Amount of work at that moment 

• Specific media and technologies required 

The business rationale for ILRC’s e- learning sfrategy 

The ILRC project was initiated as a response to the market demand. The marketing plan will depend on the 
stakeholders. It is of great importance to highlight this type of project as the first of its kind in Southern Africa. Given 
that position, there is a potential for ILRC to grow fast to neighboring countries like Angola, where Namibia already 
receives a substantial number of students, as well as to other neighboring countries like the Democratic Republic of 
Congo and form a network beyond Namibia and more so beyond Africa. Networked systems and networked 
operations management will enable the ILRC to become an international institution that can expand its services 
worldwide. 
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When the ILRC expands its wings of cooperation to other e-learning ventures, it may buy in, adapt or adopt 
courses and programs from other institutions. The networked systems will enable ILRC to respond to the needs of 
people in a timely fashion. 

The training component is the cornerstone of the project. Information and communication technologies are 
of little use without human resources, that is why the business rationale of the ILRC is training to promote local 
capacity. After graduation, the trainees will become trainers of trainers (ToTs). For sustainability the project is 
envisaged to attract some outside funds, but in the long run it has to be prepared to be nationally sustainable. The 
other stakeholders need to be selflessly involved by committing resources. 

The funding of ILRC e- learning strategy 

The initial project funded was N$ 40 million for the start of the infrastructures. Given the initial state of lack 
of resources, more funds came from Ford Foundation to furnish the ILRC, the Regional Outreach Centers (ROC) and 
the Open and Distance Learning Centers with the latest technology equipment. In addition, a National Trust Fund 
was established to keep fees for services. 

Once the donated funds ends, the ILRC will need to sustain itself. Concerning the stability of funds, the 
ILRC will depend on the marketing of ILRC Services: fund raising activities and fee for service to cover operational 
costs. 

In the long run when the grants decrease, the cost could be absorbed by the recurrent budget of UN AM 
and six Regional Outreach Centers (ROC) as well as the Open and Distance Learning Centers. In addition, the ILRC 
could be sustained by income from tuition and service fees. 

ILRC and the competitors: 

Given the fact that ILRC is hosted at the University of Namibia, a service institution, and that the equipment 
are very expensive, local companies who are attempting to sell e-learning strategies may not afford better 
infrastructures and resources. The likely competent competitors would be other educational institutions in the 
Southern Africa Region, especially South African distance education institutions that have been aggressively 
marketing themselves in Namibia. In addition, the commercial e-learning/e-business companies that are trying to 
venture into distance education may be worth considering as competitors too. 

How wiU the ILRC e-learning efforts be evaluated? 

The ILRC need a comprehensive evaluation to minimize the risks of underperformance. 

The evaluation will be done by collecting data and analyzing it in both quantitative and qualitative 
approaches. The focus of the evaluation will be on the successes and shortcomings. There shall be a control 
mechanism to establish what is working and what is not and why. The learners, instructors, technical programmers 
and staff from partner institutions will be included in the evaluation process. To avoid complacency, outsiders will 
also be required to participate in the comprehensive and ongoing evaluation of each component. 

The Advisory Board comp osed of stakeholders from partner institutions will be tasked to do a 
comprehensive evaluation during the implementation process and also arrange summative evaluation and reviews by 
outsiders as it is deemed fit. 

The challenges in the implementation of the ILRC e-learning strategy 
Stability of funding 

When the donated money dries up, it will be hard to recover the cost of the project. The rate of return will be 
slow. The ILRC will need to sustain itself. E-learning has proven to be very expensive for very formidable institutions, 
therefore it remains a challenge to sustain the ILRC and it is more difficult to run it as a real business under a service 
institution. There will be a need for money to re-engineer courses and money to pay development teams as well as 
twenty four hours, 7 days a week technical support to both learners and instructors, including content-related learner 
support services. 
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Outsourcing and vendor management 

To position ILRC in terms of outsourcing and vendor management, the project has to be viewed from the 
following spectrum of e-learning: 
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Focus on end products 



Focus on means 





Buy access 


Buy off- 


Have courses Develop 


Develop 


Develop 


to a portal 


shelf 


developed your own 


courses for 


tools for 


or training 


courses 


externally courses 


others 


others 



services 



Adapted from: Horton, W. (2001). Leading E-Learning. ASTD Publications, Alexandria, VA. 

ILRC cannot do it all, however it will be the first project of its kind in Namibia, therefore it may heavily lean 
on the left: focus on end products, at the beginning and than later reach a stage where it can focus on developing 
tools for others. 

The distribution of materials might be something to outsource at the very beginning. 

The management of vendors needs to be entrusted to those with the right expertise in the e-learning 
operations. This remains a challenge even for experienced E-Learning companies. 

Coordination of e-leaming Efforts 

The University of Namibia (UNAM) that hosts the ILRC has a Computer Center, the Interactive Multimedia 
Unit (IMMU), the Improving Teaching and Learning Unit (ITLU), and the African Virtual University (AVU) 
operations, etc. The ILRC is working in close collaboration with these campus networks. In fact it is the talents/skills 
that exist in these different units that are being tapped for creating an effective and efficient e-learning center. 
However, the most important people in this business are being left out, the lecturers/instructors. The ILRC being a 
newborn baby; some people have not heard about it, some of those who have heard do not understand the vision of 
the center. Those who are involved do not have a clear vision of where they are heading. Things are not smooth as 
there is fear of the unknown as well as fear of failure. In the process, there is a lot of competition and envy that may 
hinder the successful running of the e-learning efforts . 

ILRC gap analysis 



Gap analysis aims at highlighting the disparities between current and desired situation. 



Gap analysis 
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Gap Statement 


Root Causes 


Bandwidth 
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Upgrade to 5 12 kb 
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bandwidth & 
Internet problems 


Eunds 
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ILRC force field analysis 

Force field analysis is a way to identify factors that may inhibit the closing of the gap and the one that may 
aid in closing. 



Gap statement 


Inhibiting factors 


Aiding factors 


Inadequate bandwidth 


Availability of Funds 


Donors, fees for service 


Isolated systems created in 
isolation 


Political attitude aspects 


A positive desire to work hard so 
that all UNAM systems can talk 
to each other 


Lack of template in designing e- 
learning courses 


Lack of time to re-engineer 
courses, limited human resources 
and money 


Extra funds is a huge enabling 
factor 


Instructors are overloaded and 
regard participation in ILRC 
training as an extra load 


Promotion and career 
advancement does not measure 
their contribution to Distance 
Education (DE) 


Reward system or compensation 
model that value their 
contributions to e-learning 


Instructors are not well informed 
on the issues related to 
intellectual properties and 
copyright. 


Ignorance of underlying elements 
on human, legal, ethical and 
copyright issues 


Incentives for instructors who 
participate in staff development of 
the ILRC including copyright 
issues 



SWOT analysis 

Strengths, refers to what the ILRC is good at. 
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Weaknesses , refers to areas of underperformance or nonexistent resources that pose a risk 
Opportunities , refers to areas where progress could be made 

Threats , refers to what might happen if weaknesses overwhelm strengths and opportunities that are not realized 



SWOT analysis 


Strengths 


Weaknesses 


Opportunities 


Threats 


The ILRC is well- 
equipped with modern 
technology 


Hard to get around the 
new model of learning 
(e-learning) 


The change of 
pedagogical approach 
based on sound 
Instructional Design 
Principles 


Swimming upstream 
carrying prejudices, 
attitudes and opinions 
against e-learning 


Dedicated instructors 


Very busy instructors 
and the ILRC 
operations are not their 
priorities {Priorities are 
not their own) 


To create the right 
environment to support 
the instructors since 
they are very important 
for the operations of 
ILRC 


Traditional prejudices 
against the main stream 


Support from top 
management in the 
hosting institution and 
from stakeholders 


Lack of a unified 
understanding and 
getting around 
everyone’s political will 


ILRC to establish itself 
as an international 
leader in emerging 
technologies in 
Southern Africa 


Everybody is too busy 
to commit their time to 
ILRC endeavors 


Institutional will 


Ignorance and 
prejudices 


Ability to change the 
negative perceptions to 
the positive ones 


E-leaning enemies 



E- Learning strategies recommendations 

In preparing for e-learning strategies for such complex projects such as the ILRC, the stakeholders must 
approach the task intelligently. This paper suggests the following in order to deliver e-learning strategies 
successfully: 

a) Since the end-product will be delivered through electronics and computer technology and via the Internet, 

critical issues need to be noted such as: 

• Bandwidth Considerations 

• Multimedia considerations 

• Browser Considerations 

• Authoring Considerations 

• Content Considerations 

• Multiple -platforms consideration 

• Instructional Considerations (Bixler & Spotts, 2001). 

These issues determine the type of infrastructures needed for distributing training, learning and 
instructional materials. All the pros and cons of the media should be carefully considered for relevance and 
appropriateness to the content and environment. 

b) The information, instructional materials that might be bought or adopted should be modified or 
contextualized to meet the specific needs, provided there is expertise to do so. 

c) There will be a need for on-going support, be it technical or content related aspects. 
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d) It is not advisable to use the Internet or the WWW as the sole modes of delivery because their services may 
be limited, taking into account pressures from the demands of users. 

e) Given the fact that there are so many unanswered pedagogical questions (Shalyefu & Carr-Chellman, 2001), 
allowance must be made to accommodate different needs and various learning styles. 

f) For a program to be designed or adopted, a feasibility study needs to be done to determine the market 
demand, to define the academic products to be delivered, to design the technical model for delivery, and to 
develop the partnership needed to implement and support the program. 

g) The ILRC project is an attempt to close the gap created by the digital divide, because it increases access and 
equity. 

h) The ILRC could be an answer to the long-standing demand of Science Subjects and also for a remedial 
program needed for Mathematic Education. 

i) The ILRC Library Unit should be able to complement the print-based material with online full-text journals 
and archived materials that will be available online to learners, academics, communities and the wider public. 

j) For the ILRC to achieve success, it is essential that the project be managed by a multi-talented team of 
innovative professionals with experience in academia, distance learning, networked systems and networked 
operations management. 

k) The ILRC can also offer non-credit programs, seminars for professional development, bridging courses for 
those who want to obtain entrance qualification to the university through the e-learning resources. 

l) All ILRC staff should benefit from e-learning environments offered through hands-on training and also 
through Learning Management Systems (LMS), Knowledge Management Systems (KMS), Portals, 
Discussion Boards, etc. 

m) Technical Support Services should be able to remotely detect trouble shoots and correct a wide range of 
operational problems that occur in Distance Education networks. 
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Global Snapshots of E-Leaming: E-Leaming Status at METU in Turkey 

Tugay Tutkun 

Overview of the Country and Education System 



Geography 

Turkey is located between Europe and Asia continents surrounded by Mediterranean Sea, Black Sea and 
Aegean Sea. It is counted as a window of Western Culture opening to meet the Eastern Culture. The population of 
Turkey is 62.8 millions and the most important characteristic of the population is the amount of young population. 
Turkey is among the fastest urbanizing countries of the world and Turkish is the native tongue of 90 percent of the 
population in the country. 

The Principles of the Turkish National Education 

According to the Constitution governing the Republic no one shall be deprived of the right of learning 
education. Eurther-more, primary education is compulsory and free in state schools. Except in specially licensed 
institutions Turkish must be taught as the mother tongue and religions instruction is a compulsory subject in the 
primary and secondary school curriculum. The following are the constitutional principles underlining the Turkish 
National Educational system: universality and equality, fulfillment of individual and social needs, freedom of choice, 
right to education, quality of opportunity, and education for all throughout life, adherence to Ataturk's reform 
principles including secularism, building of democracy, scientific approach, co-education and school parent 
cooperation. 

Educational Administration 

As with the overall administration of the country, educational administration is firmly centralized under the 
Ministry of National Education. The Ministry is responsible for drawing up curricula, coordinating the work of 
official, private and voluntary organizations, designing and building schools, developing educational materials and 
so on. The Supreme Council of National Education discusses and decides on curricula, regulations etc. prepared by 
the Ministry. Educational affairs in the provinces are organized by the Directors of National Education appointed by 
the Minister. However, they work under the direction of the provincial governor. 

Einancing Education 

The major source of income of state universities is the funds allocated through the annual state budget. In 
addition to this, a university has three more sources of income. Eirst, income from the services provided by a 
university, such as patient care in university hospitals, and contract research, is collected in a revolving fund. 
Second, student contributions towards highly subsidized services are collected in a separate fund. Third, each 
university has a research fund made up of a lump sum grant from the state-provided budget plus a portion of the 
income from the revolving fund and from earmarked projects given by the State Planning Organization. 

Higher Education 

With limited exceptions, high schools are at large owned by the government and provide free educational 
opportunities. Graduates of these high schools can attend universities after the university entrance examination. The 
centralized examination is administrated by the Supreme Council of Higher Education and determines candidates for 
the enrollment of each university and faculty after evaluating the grades of related subjects, their high school results 
and their preferences according to the student capacity of each faculty. 

To expand the higher education system, preliminary studies were started in 1996 for the design and construction of a 
nationwide distance education system based on advanced telecommunication and educational technologies. Now, 
some graduate programs are in some universities are offered through internet after being approved by at he national 
committee and some undergraduate courses are also offered through internet. Talks are being held with universities 
abroad to offer joint degrees through internet. Distance education is now being seriously considered as a means to 
meet the pressing demand for higher education. 
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Culture 



Because of being the homeland of many civilizations in the past Turkey has many different cultures within 
the Anatolian Peninsula. However, from the Ottoman Empire times this population is mostly sharing the same 
religion, language and traditions with little differences. These cultures are completely mixed and formed another 
common culture after the foundation of Turkish Republic in 1923. The official language of the education is Turkish 
where 99% of people speak Turkish. However some institutions only offer education in English. 

Education has a special value in this culture, because unemployment is one of the biggest issues in Turkey 
and education is viewed as one of the solutions to this issue. Higher education is required to find a good job, 
however sometimes it is not enough and some additional skills may be needed such as computer skills, second or 
third language, etc. 

Technology 

Turkey is investing great amount of money on technology since 1980’s. In regard to communication 
technologies Turkey has made a great attack in terms of catching the contemporary technology with placing two 
communication satellites on the orbit. Computers are becoming more common day by day. Currently, about 20% of 
people have computers in their homes and about 9% of these computers are connected to the World Wide Web 
(State Institute of Statistics (SIS)). Most of these connections are dial-up connections which people could get for a 
reasonable price from several internet service providers. Other types of connections are also available such as DSL 
and broadband but, not in every region. These connections types are also more expensive than the dial-up so, people 
still prefer to be connected from their homes via their phone lines. 

METU and E-LEARNING 
About METU 

Middle East Technical University (METU) is founded under the name of Middle East High Technology 
Institute on November 15th, 1956 to contribute to the development of Turkey and Middle East countries and 
especially to train people as to create a skilled workforce in the fields of natural and social sciences. It is located in 
the capital city of Turkey, Ankara. In the academic year of 2000-2001 METU employed 706 faculty, 260 academic 
instructors and 1028 research assistants. It offers education to nearly 19.000 students and the language of the 
instruction is English ( http://metu.edu.tr/about/geninfo.t)nt) ). 

Culture 

Being a developing country, the major problem in Turkish universities is the lack of qualified faculty for 
many of the disciplines. This realization leaded the Supreme Council of Higher Education and METU to search new 
methods of transferring knowledge to wider audiences. To meet the pressing demand for higher education with the 
current human resources it is important for METU and also for the Ministry of National Education to improve and 
wide spread distance learning. To reach that goal, supported by the ministry, presidency supports and encourages e- 
learning efforts within the regulations and policy of the university and the Supreme Council of Higher Education. E- 
learning is a developing concept in the institution and it is strongly supported by authorities. 

A couple of strategies described by Rosenberg (2001, pp 185-189.) are used to build support for the e- 
learning efforts within the community such as providing easy and free access to faculty and students, paying or 
giving credit for knowledge and integrating learning to work. Access is provided both in and off-campus, however 
due to the technical difficulties off-campus access is only available by dial-up connection. Eaculties also get 
certificate and increase in their salaries if they participate in e-learning efforts. Online tutorials, information and 
discussion groups are available within the university’s website to encourage people and integrate learning to work. 
Technical assistance is also provided by the Computer Center and Informatics Institute. 
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Rosenberg’s Criteria and METU’s E-leaming Efforts 
Perception of E-Learning 

In general, e-leaning is viewed as one of the medium of distance education. Another university, Anodolu 
University, has been successfully offering distance learning courses and degrees via TV broadcasting; so, most of 
the people including faculty, students and also civics are familiar with the concept of distance education. Although 
e-learning is relatively a new concept in Turkey, the general understanding of distance learning is helping people to 
understand this emerging technology. 

Within the university management, e-learning is viewed as a resource saver; because management people 
are aware of that they could distribute university’s instruction to a wider audience with same resources in terms of 
instructor and faculty. Eaculty views e-learning as a time saver; because they think they will spend less effort for the 
online instruction than the traditional instruction. However, it seems that both the management and faculty members 
are not arguing or considering the quality of the instruction. The basic and first goal of these e-learning efforts is 
seems to be distribution of the instruction to a wider audience with the limited resources of the university. However, 
Department of Computer Education and Instructional Technology is involved within these efforts to assure and 
improve the quality of online instruction efforts. 

Current E-Learning Strategy 

In general, METU’s strategy is to allow and encourage or help and support each school within the 
university. Eor example, the Distance Interactive Learning (dlL) project offered by Department of Basic English has 
no relations with the Distance Learning Center which is supporting and managing METU’s many other e-learning 
efforts such as Asynchronous Internet Education and METU Online. dlL uses outsourced management systems 
while METU Online uses NET -Class management system which is developed by the Computer Center and 
Informatics Institute. Each department may outsource or develop their e-learning for their audience or use the 
systems provided by the institution. 

The Distance Learning Center established within the institute aims to conduct and coordinate research on 
distance learning and to monitor and lead current methods used for distance learning at and out of the university. 

The center promotes an interdisciplinary approach to research and development in distance learning. The aims of the 
center also include a systematic accumulation of knowledge and experience gained from current applications as well 
as the dissemination of this ''know-how'' at the university and to other institutions in a systematic and scientific 
manner (Distance Learning Center at METU). This center has a “Distance Education Laboratory’’ in which online 
programs are developed with the help of faculty from various departments. 

Current State of E-Leaming 

Some graduate programs are offered through internet after being approved by at he national committee and 
some undergraduate courses are also offered through internet. 

Other than departmental programs. Distance Learning Center has 4 different types of courses available 
online offered for different audiences: 

1 - Online Degree Program Courses: (Informatics Online), These courses are intended for working 

professionals who need continuing education in anytime, anywhere fashion without the need to come to the 
METU campus for lectures. 

2- Online Courses: (METU-Online), These regular online courses are offered for students where their 
performance could be assessed. 

3- Online Training Courses: These courses offer only learning resources to the students or instructors to 
provide a self-paced online learning environment without concerning assessment or any certification. 

4- Online Certification Courses: These courses are offered for professionals who are in the need of 
continuing training and education. 

Depending on the content and audience courses may have 3 different structures: online support courses, 
semi -online courses and fully online courses. Online supported courses are where lectures are conducted in face-to- 
face format with the use of such online communication tools as forum, e-mail and so on. Semi -online courses are 
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those instructions are in both face-to-face and online format, and fully online courses are actual web-based courses 
where the whole instruction is in online format. 

Current Technology 

Supported by the government to provide and manage domain names and Internet access of the country, 
METU has reach resources in terms of technology and workforce. The whole network was installed in 1990 and 
updated to a fiber optic network in 2000. The network is comprised of a number of departmental networks and various 
sizes of multi-user hosts around a campus wide backbone network. Each departmental network has multi-user and 
single user systems that are connected to a departmental network and these are connected to whole network. They 
have been providing access to this network from outside the campus since 1999. This whole network is running 
continuously 365 days of the year without any interruption ( http://metu.edu.tr/about/geninfo.pnp ). 

At METU, internet access is provided to students with the computer labs within the departments. Each 
department has at least one computer lab. Student dormitories also have computer labs with internet connection. 
Access from outside the campus is also provided via dial-up connection for free to METU students. 

Eunding 

By far, the major source of income of state universities in Turkey is the funds allocated through the annual 
state budget and METU has the same funding structure. In addition to this, METU has some more sources of 
income. First, income from the services provided by the university, such as language education, and contract 
research, is collected in a revolving fund. Second, student contributions towards highly subsidized services are 
collected in a separate fund. However, only about 1/4 of the income from student contributions goes for education, 
the rest goes to meals, lodging and medical services provided to the students, and to financing extracurricular 
activities. Within this structure e-learning efforts are founded from the university budget like any other program. Any 
income gotten by e-learning programs supported by the Distance Learning Center goes to the university budget first 
and then the presidency distributes the budget to departments and programs considering the importance of the 
services and researches provided. However, other separated and stand alone e-learning efforts which are provided 
by various departments have their own budget to found their selves within the department. 

Evaluation 

Evaluation is an aching situation. Currently, there are no serious evaluation efforts to assess the 
successfulness of e-learning efforts. Main criterion is the mass distribution of the instruction. However surveys are 
being conducted among the faculty, instructor and students to gather their opinions about the online courses. 
Academic assessment and evaluation mechanisms, eventually leading to a full accreditation system, must be 
established so that funding can be linked to performance and students or consumers can be properly informed about 
the alternatives and e-learning efforts. 

Coordination 

Since a couple of departments are participating in METU’s e-learning efforts, coordination between these 
departments becomes so important. The Distance Learning Center has been founded to manage these coordination 
tasks. However there are some other e-learning efforts which are completely separated from Distance Learning 
Center. Those separated efforts bought their management systems from outside while DLC is providing a free 
management system. This reflects lack of coordination between the departments. In general, METU’s various e- 
learning efforts are not properly coordinated within the institution. Although these separated e-learning efforts are 
commercial while others are for METU students there is still a need to coordinate these efforts. It may be a good 
suggestion to coordinate entire efforts as commercials and non-commercials, so that management of these systems 
becomes easier. 

Competition 

E-leaming is a developing concept in Turkey and education is mostly provided by government institutions. 
There is a lack of big companies or institutions offering certified online courses or degrees so, METU does not have 
any serious competitors. METU is basically taking the advantage of being a government supported institution and 
its reputation which is coming from their successes on traditional courses and certificate programs in the past. On the 
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other hand, since METU is one of the leading universities in Turkey, their certificates are more popular and desired 
than any company or institution providing e-leaming. Most of the companies require METU certificate for 
employment. They do not need to compete for their non-commercial e-learning products, because they only offer 
these courses to their students for free; just a replacement of traditional courses. 

Instructional Development Capabilities 

Within the Eaculty of Education METU has the Department of Computer Education and Instructional 
Technology which is participating in DEC e-learning efforts in terms of providing instructional design knowledge. 
This department is studying on emerging technologies and their use in education. 

On the other hand. The Distance Learning Center is continuously conducting researches on distance 
learning. They also have connections with the universities abroad in terms of sharing knowledge and technology. 
Because of being a large university, METU has various resources from many different departments. Eor example. 
Computer Center and Informatics Institute is providing technological resources while the Department of Computer 
Education and Instructional Technology provides instructional resources and knowledge. The entire e-learning 
efforts within the Distance Learning Center are a collaborative study of the various departments. 

Since the e-learning is important to the country The Supreme Council of Higher Education and METU have 
been sending graduate students abroad in the related fields to come back and work at METU. These returning 
graduates will not only enhance the current efforts but also help to improve the vision of e-learning. So, the 
propensity for sustaining e-learning will continue to increase in next decades as long these efforts supported by the 
authorities. 

Conclusions 

In general, METU has been successfully offering online courses and degrees. However, there are some 
areas that needs to be improved, such as perception of e-learning, evaluation of the efforts, coordination and 
funding. Especially perception of e-learning should change from “a medium to distribute the content to wider 
audience” to “a medium to offer richer and better instruction to wider audience”. After analyzing the current situation 
and describing the desired situation that an institution should have I may report a couple of ideas that METU could 
DO or NOT. 

DO’S 

• Centralize entire e-learning efforts, which make them easier to manage. In this case it will also save some money, 
in terms of not paying for the management systems. 

• Develop an actual and effective evaluation strategy to continuously improve e-learning efforts. 

• Develop a performance based funding strategy, so every department could take what it deserves. 

• Improve the coordination between the Distance Learning Center and Departments and also Eaculty. 

• Continue to improve accessibility of the internet and online courses. 

• Continue to follow-up new technologies and knowledge at other universities and companies, which are offering 
e-learning, around the world. 

DO NOT’ s 

• Do not only focus on reaching more learners, but also quality of the instruction and content. 

• Do not let technology to become a barrier between learners and the content. 

• Do not expect people to accept and support e-learning, continuously help them to understand the rationale and 
benefits of e-learning. 
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Because the complex link and node structure awaiting users can lead them into becoming lost in hyperspace and 
cause them cognitive overload, navigating the hypertext system is often not an easy task, especially for novices. 
They have a difficulty perceiving a structure a structure of entire system, locating specifying information, using 
navigational aid. This phenomenon can be more expected in the highly structured EPSS. In that point, content 
providers or designers for web-based integrated EPSS should know how to design interface and information 
structure based on its content and purpose. 

This study analyzed four interface design methods (simple selection menu style, menu with global navigation, menu 
with global and local navigation, and pull-down menu) and four information structures (linear structure, grid, 
hierarchy, and network) in terms of the complexity, flexibility, navigation, domain knowledge, and cognitive load. 
Based on this analysis, guidelines for building the integrated web-based EPSS effectively are provided. 

Introduction 

Building an effective EPSS has been a major issue not only in business but also in education and 
government. As more people come to depend on the use of computers and networking to perform their jobs, and as 
hardware and software technologies continue to advance, the potential for EPSS appears tremendous (Collis & 
Verwijs, 1995; Malcolm, 1992; Ockerman, Najjar, & Thompson, 1997). 

Hypertext and hypermedia system enabled designers and programmers to tile all performance support 
system elements together in a way that allowed users to follow their own streams of thought in searching for 
information (Gery, 1991). Hypertext is such a technology that provides a powerful new way of organizing, displaying, 
and accessing information that could affect all forms of systems (Shneiderman & Kearsley, 1989). Hypertext consists 
of the associative links between multiple nodes, which are one or more parts of information, forming an 
interconnected networked (Conklin, 1987; Nelson, 1974). The linking system in the hypertext system allows users to 
browse through the system utilizing navigational tools (Eklund, 1995). However, navigating the hypertext system is 
often not an easy task, especially for novices (King, 1996). The potentially complex link and node stmcture awaiting 
users can lead them into becoming lost in hyperspace (Nielsen, 1990b) and cause them cognitive overhead (Conklin, 
1987). Users have a difficulty perceiving a structure of entire system, locating specifying information, using 
navigational aid. This phenomenon can be more expected in the highly structured EPSS. 

Such problems have prompted research on the manner in which users interact with hypertext system. Usable 
design guidelines and principles for navigation can maximize coherence of integrated EPSS and minimize users’ 
cognitive overhead and disorientation. 

Interface design 

When designing an integrated EPSS, two important issues are involved: 1) interface design - how to present 
link system that provide access to the structure, and 2) information structure - how to incorporate the original 
structure of the content into the structure of an EPSS. 
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Interface design is basically concerned with the presentation of text, graphics, and linking system on the 
screen. It provides a contextual or structural model for the specification of the logical and functional organization of 
the user interface component, as well as a communication and means between users and system (Laverson, Norman 
et al. 1987; Norman and Chin 1988; Lai and Waugh 1994; Oliveira, Goncalves et al. 1999). Many researchers has been 
agreed that interface should be designed to provide users efficient and effective organizational model that can help 
users understand the entire system and navigate system to find information without getting lost or cognitive 
overload (Dieberger, 1997; Lai & Waugh, April 1994; Schenkman & Jonsson, 2000; Shneiderman, 1997). 

Interface on an EPSS provides three major roles for navigation; 1) presenting links, 2) supporting structural 
cues, and 3) providing path mechanism. The interface is the most immediately visible part when a system was user- 
centered (Nielsen, 1990a). The most fundamental function of interface is to display links one the screen so that a user 
could navigate through a system (Shneiderman, 1998). Selecting the links is frequently difficult to navigate in spite of 
the fact that a graphical browser provides the easy to navigation method, “point and click.” One factor of difficulty in 
selecting paths for navigation is the presentation of link system. The user navigation performance can be influenced 
by the design and placement of links (Carlson & Kacmar, 1999). 

The second role is to support the user perception of structural cues for an effective navigation. If the 
interface does not provide appropriate structural information, users cannot perceive where they are and cannot 
decide where to go. Users also could experience “lost in space” (Dieberger, 1997). 

Information is structured by links and users follow paths developed by designer. As a result, if interface 
design provides the flexible path mechanism that can allow users to jump to the information directly without passing 
pages that are not necessary to get the information, users could find information fast. However, there is some 
possibility to increase user’s cognitive load without providing structural and navigational cues. 

There are four generally used interfaces for navigating the web site: 1) simple selection style, 2) global 
navigation aid, 3) global and local navigation, and 4) pull-down menu styles. 

1) Simple Selection Menu Style 

One type of menu style is simple selection menu style, which is much similar to a table of contents for a print 
book (Chimera & Shneiderman, 1994). In this menu style, a user can go to deeper levels of the web site by selecting 
links presented in the current web page. The major drawback of this menu style is that a user has a difficulty in 
perceiving the entire structure of the web site (Chimera, 1994), since when a user moves into lower levels, previous 
menu is replaced with new level. Another drawback of this menu style is that a user is unable to traverse to beyond 
below level because a user has to wait for a new below level select link again before moving lower level and select the 
link below level again. An experienced user is unable to navigate faster than that that of novice and this is inefficient 
and frustrating for experienced users (Laverson, Norman, & Shneiderman, 1987). 

2) Global Navigation Aid Menu Style 

The second menu style is persistent menu that has two split parts. The links on the top level remain in the 
similar area on the left all the time. The content is located in the area on the right and is replaced by its subsequent 
menu when users move other pages. The advantage of this menu style is that a user can go to each page of the top 
level by clicking top level links on the left side from any page in the web sit and that it provides the global structural 
cue to users (Nanard & Nanard, 1991). 

3) Global and Local Navigation Menu Style 

The third menu style has two parts of navigation link for global level and local level (Nanard & Nanard, 
1991). The navigation links of top level pages on the web site usually place on the top of web page and the 
navigation links of current level pages on the web site usually place on left side. The advantage of this menu style is 
to provide not only global structure cue with the top level navigation links but also local structure cue with current 
level navigation links (Furnas, 1997). A user can skip several levels in the web site and it can more efficient than other 
menu design because a “jump -ahead” capability can reduce the time to navigate and find information (Laverson et al., 
1987). 

4) Pull-Down Menu Style 

Pull-down menu style appears over objects in the interface instead of in static menu area. Pull-down menu 
allow users with a mouse to access the Web page they want directly. The advantage of the menu style is jump to the 
any page with mouse move and click. However, users can be disoriented or get lost since this menu can’t provide 
structural cues. This menu style would be useful for experienced users. 
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Table 1. Summary of Menu Style 





Use 


Complexity 


Flexibility 


Access Speed 


Disorientation 


Simple Selection 


Easy 


Low 


Low 


Slow 


Low 


Global Navigation 


Easy 


Middle 


Middle 


Middle 


Middle 


Global & Local Navigation 


Hard 


High 


High 


Fast 


High 


PuU-Down 


Middle 


Very High 


Very High 


Very High 


Very High 



Information Stmcture 



The structure of a hypertext system can take many different forms. Four basic structures are linear, grid, 
hierarchical, and network stmcture. However, a web-based integrated EPSS can be designed with more than one 
structure. 

1) Linear Structure 

The simplest way to organize information is in a sequence based on chronological or logical orders. 
Typically it is useful structure to retain the original documentation. Linear structure can be used for guided tour, job 
aids, tutorials, and demonstration of procedure in Web-based EPSS. 




2) Grid 

Grid structure organizes multi-dimensional concepts or categorizations. A serous of procedural manuals and 
list of training courses and materials can be best organized. Grid structure can be organized with one concept or 
categorization in horizontal axis by vertical axis with other concept or categorization. Unfortunately, grid structure 
can be difficult not only to organize but also to understand unless the designer or user recognizes the 
interrelationships between concepts and categorizations of whole information. Therefore, it is best for who already 
have knowledge on topics and its organization (Lynch & Horton, 1999). 




3) Hierarchy 

A hierarchical structure bas been used widely in the web site (Lynch & Horton, 1999), on-line 
documentation (Gloor, 1997), information retrieval system (Rosenfeld & Morville, 1998), and computer-based 
instructional programs and training (Jonassen, 1986; Lai & Waugh, 1994). Organization of hierarchical stmcture starts 
with general concept or topic into specific ones, which are in turn divided into more specific to individual based on 
precedence and significance (Lynch & Horton, 1999; Norman & Chin, 1988). Users can move from general to specific 
and back to general through the linking system presented by menu design. The advantage of hierarchical structure is 
that familiar to most people since it reflects the stmcture of printed materials (Shneiderman & Kearsley, 1989), it is 
ubiquitous in everyday life (Sand, 1996), and it is the most natural structures for organizing levels of abstraction 
(Gloor, 1997). Because of its familiarity and pervasiveness, users can easily and quickly understand web sites and 
hypertext system (Rosenfeld & Morville, 1998) without heavy cognitive overload. In organized hierarchical 
information structure, the disorientation problem can be minimized and users can easily navigate among information 
nodes by following the linking system (Conklin, 1987). They are able to develop a mental mo del of the site’s structure 
and their location within that stmcture (Rosenfeld & Morville, 1998). 
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4) Network 

Network structure is composed of associative links that network related concepts and information together. 
While this structure provides relatively effective navigation mechanism, users can easily get lost or disoriented 
because it is so hard for users to understand and predict entire. This structure can apply on small size of Web site for 
high level of training, such as simulation game, strategy training. It would be difficult to manage linking system if size 
is getting bigger (Lynch & Horton, 1999). 




Most complex web sites share aspects of all four types of information structures. Except in sites that 
rigorously enforce a sequence of pages, users are likely to use any web site in a free-form ''web-like'' manner, just as 
most non-fiction or reference books are used. But the nonlinear usage patterns typical of web surfers do not absolve 
developers of the need to organize their thinking and present it within a clear, consis tent structure that complements 
design goals for the site. Table 2 shows the characteristics of four basic organization patterns explained above in 
terms of the complexity, flexibility, easy-to-navigation, domain knowledge, and cognitive load. 



Table 2. Summary of Information Stmcture 





Complexity 


Elexibihty 


Navigation 


Domain Knowledge 


Cognitive Load 


Linear 


Low 


Low 


Very Easy 


Low 


Low 


Grid 


High 


High 


Middle 


High 


High 


Hierarchy 


Middle 


Middle 


Easy 


Middle 


Middle 


Network 


Very High 


Very High 


Hard 


Very High 


Very High 



Depth and Breadth 

The most discussed issues in information structure are branching factors, depth (the number of vertical 
levels or links) and breadth (the number of horizontal documents or links. Much has been known about tradeoff of 
“depth and breadth”, and “complexity and flexibility issues(Shneiderman, 1997). The previous results generally 
support that deeper hierarchical structure, the less helpful and less favorable for navigation. Users have to move 
through more levels to located information in the deeper structure and should need more not only cognitive effort but 
also physical effort to interact with more displays and links for information searching (Shneiderman & Kearsley, 

1989). 

Design strategies for building an integrated EPSS 

The effective design strategies for building an integrated EPSS can be affected by many factors. These 
strategies include as follows: 



1) Analyzing nature of information or knowledge 
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In building an effective Web-based integrated EPSS, one of the most important things developers have to 
keep in mind is to analyze the nature of information or knowledge to be supported by the site as exactly as possible. 
Depending on the professional knowledge with a complex information structure or the general knowledge with a 
simple information structure for its delivery, it might be totally different in terms of the interface design, information 
structure, and the depth and breath of the information. 

2) Analyzing users’ characteristics 

In order to build an effective Web-based integrated EPSS, developers should clearly analyze the 
characteristics of the target users of the site. That is, the developers need to conduct a more detailed analysis of the 
target audiences, such as teenagers, professionals of some fields, and so forth. Based on that analysis, the 
developers can grasp the major concerns of the audiences and predict their preferences. This will lead to providing 
appropriate functions just in time when such needs arise. If the EPSS is developed without seriously considering the 
users’ characteristics, including their preferences, the site, no matter how well organized or useful, will not appeal to 
the some portion of the users anymore. 

3) Localizing contents and categories 

Due to the rapid development and spread of communication technology, traditional borders disappeared in 
terms of information sharing. In other words, we can share any content as soon as it is developed. Considering the 
importance of sharing information, this is very desirable. The problem arises for the fact that every country has 
unique cultural and social context as well as different language. Even in the same country, there can be various 
cultural groups. Certain problems are expected in the use of certain contents in different countries or cultural groups. 
Eor example, item 'I', which is categorized as 'A' in one country, can be categorized as 'B' in another country. There is 
a need to readjust contents or categories in terms of cultural and social contexts. The easiest solution is to prepare 
different versions for different groups. Here, the developers may most likely face the effect of tradeoff due to 
additional cost. 

4) Analyzing client system capability 

The developers need to be cognizant of the system environment under which the users are learning. This 
factor is closely associated with analyzing users’ characteristics. While the latter is more concerned with physical 
and affective characteristics of the users, the former is more focused on their learning milieu. With the rapid 
development of technologies, we have recently seen many products with newly -added features. Even the products 
marketed in the same year maybe different from one another in terms of system capability. In this vein, the developers 
need to continue their analysis on the learning environment of the users. 

5) Analyzing information accessibility 

We cannot overemphasize the importance of information accessibility for building an effective web-based 
integrated EPSS because the EPSS is delivered via electronic telecommunication technology like Internet. While most 
users with intent to access to Internet use LAN in their office, they rely on modem at home. So, their major concern is 
how expeditiously and conveniently they can access the needed information. When they cannot obtain the 
information they need at their convenience, they will experience inconveniences arising from the use of the site. This 
will eventually turn the users away from the site. Therefore, high level of information accessibility is very important 
factor of a successful EPSS. 

6) Planning information updatability 

Information updatability is another critical issue. In EPSS, this factor is more important than any other fields, 
because the users want information, which is most recently updated at their convenience. In this regard, two most 
important issues arise as to how promptly the information can be updated and who will assume the role of updating 
the information. 

7) Harmonizing internal relationships between other information sources 

Information in EPSS is sophisticated rather than fragmentary. As seen in Eigure I, even the same types of 
information can be restored and delivered in various ways. Some information can be delivered by visual formats such 
as text or graphics or by audible formats such as audio. In this sense, harmonizing internal relationship between other 
information sources is very important. If the internal relationship is not well organized, the EPSS is just an archive of 
unrelated, individual information. 

8) Integrating external relationships between other EPSS 
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Finally, integrating external relationships between other EPSS is another important issue. The developers 
should consider that the EPSS has to be related to external sources at some degree in order to maximize its 
effectiveness. Just as harmonizing internal relationship between other information sources was important, integrating 
external relationship between other EPSS in a harmonious way is a very important factor of successful EPSS. 

All of these factors impact the way to present and structure the web-based integrated EPSS. In order to 
maximize the usability of web-based integrated EPSS, therefore, we need to identify and select appropriate menu 
systems and information structures. 




Eigure 1. Overall Structure of Web-based Integrated EPSS 
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Using Imperceptible Digital Watermarking Technologies to Transform Educational 

Media: A Prototype 

Tammy M. McGraw 
Krista Burdette 
Virginia B. Seale 
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lAETE at A RT. 



The Institute for the Advancement of Emerging Technologies in Education (lAETE) at AEL recently 
explored the potential benefits and limitations of traditional print-based textbooks and many e-book alternatives. 
Having considered these media, lAETE created prototype interactive textbook pages that retain the salient aspects of 
print media while providing access to unlimited electronic resources. While the prototype appears to be a standard 
textbook, imperceptible digital watermarks embedded in the graphics act as convenient conduits to electronic access 
of dynamic content, assessments, applications, and communication tools. lAETE is exploring digital watermarking as 
an alternative to delivering electronic instmctional media. 

Traditional Textbooks 

Textbooks have long been the mainstay of instructional media in classrooms; however, much has been 
reported about the inadequacies of widely adopted print-based textbooks. In a review of 12 prominent physical 
sciences textbooks, nine reviewers (seven of whom are professors in physics departments in various institutions of 
higher education) determined that none of the books were adequate and most contained hundreds of errors (Hubisz, 
2001 ). 

Project 2061, the science, mathematics, and technology education reform initiative of the American 
Association for the Advancement of Science (AAAS), reported similar findings. A Project 2061 study examined how 
well middle-grade textbooks help students learn key ideas in earth science, life science, and physical science. Dr. Jo 
Ellen Roseman, who headed the study and now directs Project 2061, stated, “This study probed beyond the usual 
superficial alignment by topic heading. Instead, it examined the texts’ quality of instruction, using criteria drawn from 
the best available research about how students learn” (Project 2061, 1999). The evaluation process was developed 
by scientists, mathematicians, educators, and curriculum developers, with funding from the National Science 
Eoundation. Review teams included teachers, curriculum specialists, and professors of science education. The 
review found none of the widely used science textbooks for middle school to be satisfactory (Project 2061, 1999). 

Another criticism of traditional print-based textbooks is that they quickly become outdated. Chris Jonson, 
social studies editor for McDougall Littell (a division of Houghton Mifflin), noted in a National Public Radio interview 
that production of print-based textbooks immediately stopped following the terrorist hijackings and attacks of 
September 11, 2001 (National Public Radio, 2001a). Textbook companies understood the importance of including in 
their textbooks the events of that day and the days that followed. Textbooks without such information are now 
outdated. Given the limitations of current print-based textbooks and the opportunities created by digital 
technologies, conditions appear to be favorable for the introduction of electronic alternatives to supplement, if not 
replace, traditional textbooks. 

Electronic Books 

Electronic books come in multiple formats, and the term electronic book is broadly defined in the industry. 
Some use e-book to refer to the electronic delivery of what was once presented as text printed on paper. The user 
reads text on a computer display, often a laptop computer or handheld device. Used this way, devices such as 
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Compaq’s iPAQ can be classified as e-books, though, like laptop or desktop computers, they are rarely used solely 
for this purpose. 

Some people use the term e-book to refer to specialized electronic devices modeled after the familiar print- 
based book. These devices, ranging in price from $125 to $600, typically employ proprietary file formats and serve a 
single purpose. 

Adherents of multipurpose, handheld computing devices, such as the Palm handheld or Compaq’s iPAQ, 
think these will be the most prevalent e-books for classroom use. Others propose that software, such as Microsoft 
Reader or Adobe Acrobat, will form the basis of the most viable electronic book model. In short, promoters of 
electronic books take many approaches to the production and dissemination of information. 

Issues Influencing E-book Development 

Perhaps the lack of standardization — or even suitable parameters in e-book technology — and the absence of 
a clear leader in the market have contributed to a flurry of activity in the publishing industry. Many companies are 
trying to position themselves to set the standard for others to follow. Certainly standards have been implemented to 
provide a consistent basis for e-books such as goReader and similar products. The Open eBook Forum (OeBF) was 
developed by the Association of American Publishers with assistance from companies such as Microsoft and Adobe 
(http://www.openebook.org). At the core of the OeBF effort has been a concern for safeguarding intellectual 
property and protecting current economic models (Jensen, 2000). This concern has not necessarily impeded 
progress. Looney and Sheehan (2001) suggest that digital rights management of software and electronic commerce 
has been a catalyst for e-book interest. 

Although considerable attention focuses on standards that may make e-books more attractive to both users 
and content creators, other concerns have emerged. For example, Nielsen (2000) suggests that users read 
approximately 25 percent slower from a computer screen than from a printed page. While handheld devices are 
proliferating in schools, they are being deployed without regard to such issues. This impact this might have on 
learners is clearly no small matter. According to Morehead (2001), Kathleen Brantley, director of product 
development at Market Data Retrieval, suggests that it is still very early to quantify the number of handheld devices 
currently in use in schools. Yet Palm reports that education is approximately eight percent of its business 
(Morehead, 2001). While the actual number of handheld devices currently in use in schools is difficult to determine, 
Zeitchik, Reid, and Nawotka (2001) estimate there are now more than 12 million personal digital assistants in 
circulation. 

Another concern involves the fluidity of e-book content. For example, what appears to be a benefit in the 
wake of the events of September 11, 2001 — the ability to customize content quickly — might also present problems. 
Gilbert Sewall, director of the American Textbook Council, notes that local pressures could influence how sensitive 
topics (e.g., evolution or the depiction of Columbus) are presented (Axelson & Hardy, 1999). Customization of 
content, then, could lead to vast curricular differences among schools, making student transfers, even between 
schools in the same district, more difficult. 

Policy issues have also been raised. Textbooks are typically adopted for a predetermined number of years, 
and the fact that important events that should be included can and do occur soon after adoption is regrettable but 
understood. Conversely, one of the most attractive features of electronic books — their ability to be updated 
regularly and quickly — then creates additional work for personnel responsible for textbook adoption. Furthermore, 
new funding models (e.g., subscription-based services versus one-time purchases) must be taken into account. It 
appears that electronic textbooks could save schools money, as traditional text printing and storage costs are 
significant. However, this benefit may not be immediately apparent. According to Odlyzko (1999), publishers 
generally encounter additional costs — not savings — to produce electronic publications. 

It is difficult to justify eradicating printed textbooks from the options available to schools. Despite 
inadequacies, McKnight, Dillon, and Richardson (1996) point out that, “We have had nearly 500 years experience of 
using printed textbooks, and they not only support a wide range of applications, but users also have such a strong 
mental model of their generic structure and organization that they can successfully adopt an equally wide range of 
usage strategies’’ (p. 631). They further suggest that while hypertext can support activities that would be difficult, if 
not impossible, to accomplish with printed text, we must be sure that these capabilities are used in pursuit of valid 
learning tasks: “It is not sufficient that we can browse a million pages on our desktop, or link 100 articles together for 
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rapid retrieval at the click of a mouse button: such capabilities are only important in terms of their utility to human 
learners. Yet there are few signs that most learning scenarios require such support, and little knowledge on how we 
might best provide it in terms of usability, even if it were required” (p. 631). 

In Search of the Best Alternative 

It is hard to predict what textbooks might look like in 5 to 10 years. During an interview, Preston Gralla, 
technology author and executive editor of ZDNet, proposed that education could be a niche market for electronic 
publishers, regardless of their success or failure in other areas (National Public Radio, 2001b). Indeed, major textbook 
companies are positioning themselves to take advantage of the surge in portable handheld devices in the education 
market. Strategies differ from company to company, but three approaches are most common: handheld devices, 
customized printing, and Web-based delivery. 

Handheld Devices 

Harcourt College Publishers is partnering with goReader to offer electronic college textbooks, a move that 
signals their support for the e-book platform (Writers Write, Inc., 2001). McGraw-Hill is partnering with Digital Owl to 
“mobilize and market digital information through targeted Web sites and handheld devices” (McGraw-HUl 
Companies, n.d.). 

Customized Printing 

Classwell Learning Group, in partnership with Kinko’s, exemplifies a customized publishing approach. 
Teachers can access online tools and content supplied by Classwell to create customized workbooks for students. 
Once complete, the workbooks are electronically delivered to Kinko’s, which prints, assembles, and delivers them 
(Classwell, 2001). Other customizable publishing efforts include McGraw-Hill’s Primis Online, which provides access 
to more than 350,000 pages of content, as well as PowerWeb sites that draw on the Internet’s power to provide 
current information (McGraw-Hill Primis Custom Publishing, n.d.). 

Even as customized publishing makes it possible to provide content to meet the individual needs of learners, 
proposed legislation promises to make textbook content delivery more accessible for all learners. The proposed 
Instmctional Materials Accessibility Act of 2002 would mandate that students who are visually impaired or otherwise 
impeded by print media have access to instructional materials, including textbooks, in formats they can use 
(American Foundation for the Blind, 2001). With the establishment of a national clearinghouse to receive, catalog, 
store, and disseminate electronic files from publishers, the benefits are potentially far-reaching. 

Web-based Delivery 

Much of the current activity around e-book development focuses on Web-based delivery of content, 
though approaches vary tremendously. Many of the major publishers, including McGraw-Hill, Pearson, Thomson, 
and John Wiley, have added multimedia and assessment supplements to their textbooks (Zeitchik, Reid, & Nawotka, 
2001). McGraw-Hill’s comprehensive approach provides e-text that mirrors the book’s print edition and incorporates 
features such as audio and video clips. To complement this effort, the company has launched the McGraw-Hill 
Learning Network, offering a number of resources for educators, students, and parents (Trotter, 2001). Houghton 
Mifflin and netLibrary have launched a digital textbook initiative aimed at bringing Houghton Mifflin’s College 
Division texts to a Web-based platform called MetaText (“Textbooks to Become Interactive,” 2001). 

The :CueCat bar code technology, another Web-based approach, is being used by Harcourt College 
Publishers in more than 80 college textbooks. A cat-shaped scanning device is used to swipe bar codes from the 
printed page and take the reader’s computer to a specified Web site. Houghton Mifflin is also piloting the 
technology in some college-level textbooks (“Harcourt Embraces :CueCat Technology,” 2001). 

The Interactive Textbook: Combining Old and New 

In designing next-generation textbooks, lAETE staff think it only practical to consider what McKnight, 
Dillon, and Richardson (1996) describe as our most successful information technology — the printed book. To 
address the inherent limitations of print media, the application and implications of imperceptible digital watermarking 
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technologies were considered in the context of learning environments. Prototype textbook pages were developed 
using Digimarc MediaBridge software and a widely used middle school science textbook. 

The software creates an imperceptible digital watermark consisting of XML-based code embedded in a 
graphic or other media element. With the aid of an optical reader device, such as a digital camera connected to a 
computer, the software “reads” the watermark, activates a standard Web browser, and delivers the user to a specified 
Web site or application. The interactive textbook extends the functionality of print media so that applications now 
possible on the Web, including discussions with peers or experts, real-time data manipulation, and participation in 
simulations or virtual environments, are possible within the context of the interactive book. 

Static information, such as the definition of a term, is presented in a common textbook format and 
organizational structure, but information likely to change, such as an assessment tool, is linked to the printed page 
with an imperceptible digital watermark. The digital watermark feature makes the content highly customizable. 

Unique identifiers embedded in each image enable a content manager to access or change the referent URL to 
provide a relevant and timely experience for the user without requiring any change to the printed book. 

Embedding a watermark in an image is as easy as applying a filter in Photoshop. The process for 
transforming a printed textbook into an electronic textbook does not displace well-established textbook production 
methods and utilizes standard image processing software. It is already adopted by print media and could be easily 
integrated by the textbook publishing industry. 




Figure 1 . Prototype pages developed by lAETE at AEL 



Prototype pages are presented in Figure 1. An imperceptible digital watermark is embedded in each graphic, 
delivering users to a specified Web site or application. The resource bar on the left contains five resource links: 



1- Glossary links users to definitions of terms related to the subject content and can be chapter- or even 
page-specific. 

2- Ask an Expert links users to experts who can answer student-generated questions. The expert could be 
identified by the textbook company or by the teacher, allowing international, national, local, or school- 
based authorities to share information. 
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3- Did You Know provides a connection to materials designed for differentiated instruction. Students who 
progress quickly through their work could explore many layers of supplemental content through this link. 
The activities could be high-interest and require varied skill levels. 

4- Other Resources presents supplemental, content-related materials, information, and activities. 

5- Check for Understanding offers opportunities to assess student comprehension through traditional 
assessments or problem-based assignments. 

The tool bar on the right displays nine icons linked to different tools (i.e., Sound Tools, Concept Map, 
Database, Text Editor, E-mail, Graphics Editor, Presentation, Spreadsheet, and Web). Each icon provides a shortcut 
to a designated application, giving students instant access through the watermark in the textbook. 

The illustrations on the page are also digitally watermarked and serve as a point of departure for activities 
related to the text. In Eigure 1, the digital watermark in the “Mystery Bacteria?” (ear) illustration delivers students to 
an online virtual lab simulation where they attempt to identify the pictured bacteria. Other possibilities for links 
include labeled diagrams, enhancements of photographs or maps, and questions related to the illustration. 

The adaptive nature of the technology holds promise for highly customizable content in multiple curricular 
areas. Differentiated instruction is supported by enabling students to “drill down” in content. Real-time, embedded 
assessments are supported, as is the ability to follow a student’s path through the content. Sound tools, concept 
mapping, and other applications engage students and enable them to interact with the content in whatever way is 
most appropriate for them. Although the examples above were used in the prototype, different tools and resources 
could easily be used to support other content areas. 

Usability Study and Group Interview 

Eollowing the development of the prototype, lAETE conducted a usability study, then a small group 
interview. Participants included middle school and high school science teachers. The usability study was conducted 
prior to the focus group to elicit unbiased results. The participants were given five assignments, without instruction, 
which required participants to access multiple resources and tools. They worked as a group, and a talk-aloud 
protocol was used. Eollowing the usability study, the group discussed the prototype and identified five key benefits: 
(1) the availability of real-time information, (2) more efficient computer use, (3) accommodation of multiple modes of 
learning, (4) standards-based content, and (5) the availability of assistive resources. Eour limitations were noted: (1) 
access to computing device, (2) cost, (3) durability, and (4) the need for additional workspace. 

The teachers rated the interface design and the ease of operation of the interactive textbook using a five- 
point scale, where 1 was the most negative response and 5 was the most positive response. The mean response for 
interface design was 4.75, and the mean response for ease of operation was 3.5. 

During group discussion, the tool bar was the subject of much debate. Some teachers thought it was 
unnecessary, while others argued that the shortcuts would be helpful for keeping students on task. The teachers 
brainstormed about the possible layout (e.g., putting the tool bar on the back cover) and potential functions (e.g., 
using the sound tool to assist ESL students). lAETE will use the results of this study to refine the prototype and 
prepare for a larger usability study that will include teachers and students. 

Summary 

At present, e-books appear to be supplementing print media, not replacing them (Ditlea, 2000). This 
suggests that print-based textbooks are likely to be prevalent in classrooms for some time. This prototype concept, 
using imperceptible digital watermark technology, demonstrates a realistic and practical method for linking print and 
digital media. It is likely that the limitations identified through the usability study and focused group interview can 
easily be addressed as handheld computing devices become more commonplace in classrooms and more suitable 
optical reader devices become available. 

Imagine it — a smaller textbook accompanied by a small, wireless, tablet-like computer and reader 
device/stylus that is no bigger than a permanent marker. The interactive textbook is a viable model affording 
students the best aspects of print and electronic media. 
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As we enter the twenty first century, it is essential that the schools be places that help students better 
understand the complex, symbol-rich culture in which they live in. A new vision of literacy is essential if 
educators are serious about the broad goals of education: preparing students to function as informed and 
effective citizens in a democratic society; preparing students to realize personal fulfillment; and preparing 
students to function effectively in a rapidly changing world that demands new, multiple literacies. Renee 
Hobbs, 1997 

Abstract 

Semiotics is one of the approaches to media education and new media literacy. It complements and promotes 
the ideals of Media Literacy education. It opens a new way to the study of media literacy. We can not only study the 
alphabet of deaf-mutes, symbolic rites, polite formulas, military signals, and religious symbols but also study the 
commercials, sit -corns, soap operas, bulletin boards, and now the Internet. In a sense, we all are using, 
deconstructing, learning, and reading signs even though we never studied Semiotics. We learn new signs everyday 
as the signs evolves continually in their meaning or significance all the time. 

In the millennium, we are going to be surrendered by more and more images from bulletin boards to Internet, 
from advertisements banners to book covers. Internet and new technologies create new images, icons, symbols, and 
metaphors to study for us to make sense on the Cyberspace. 

Instead of consumers of media, new generations need to be the producers of media. The more they study 
and experience media, the more likely they see, understand, and interpret media. Len Masterman (1985) considers 
media literacy education is a crucial step towards “participatory democracy.” Masterman adds, “Media Education is 
both essential to the exercising of our democratic rights and a necessary safeguard against the worst excesses of 
media manipulation for political purposes.” 

The focus is to highlight the importance of studying signs, symbols in media education in order to gain 
different perspectives in our teaching styles and strategies, instead of going into the details of the history of 
semiotics and media studies. 

In conclusion, the main goal of this presentation is to draw on the natural links between semiotics and media 
education. We will see how a critical approach to the study of media combines knowledge, reflection, and action, 
promotes educational equity, and prepares new generation to be socially responsible members of a multicultural, 
democratic society. 

Introduction 

This paper is based on a comprehensive exam topic entitled. Semiotics of New Media Literacy. It was 
presented in the School of Education at the University of Massachusetts at Amherst. The focus of the paper is to 
highlight the importance of studying signs in media education in order to gain different perspectives in our teaching 
styles and strategies, instead of going into the details of the history of semiotics and media studies. 

The paper defines “Semiotics” and “Media Literacy,” examines digital media and its culture in education 
from a semiotics perspective, and provides resources and examples for integrating semiotics and new media literacy 
into the curriculum. 

The PowerPoint Presentation of this presentation is online at: 
http://eut)hrates.wpuni.edu/facultv/vildizm/AECT2002.t)t)t 

The web page also includes links to articles, journals, organizations, and various resources on the Internet. 

Semiotics 
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Semiotics, the science of signs, is a type of scientific inquiry that studies virtually everything we do, use to 
represent the world around us and to make messages about it. Semiotics or semiology is considered a subject, a 
movement, a philosophy, or science. The example of the color (and the word) red is a sign in semiotics. The 
semiotician refers to the meaningful location of this specific light-frequency property as context, to its meaning in 
specific contexts as signification, to the ways in which it generates meaning as a code-based, and to the ways in 
which a message is understood as interpretation. (Danesi, 1994) 

The subject matter and the key concepts of semiotics has always been the 'sign'. Semiotics studies every 
sign, icon, or symbol from trafic lights to charts and graphics on a newspaper, from commercial advertisements to 
icons on the web pages, from gestures to dance movements. 

Interest in signs is not recent. Aristotle, Hippocrates, Locke and many others have contributed to semiotics. 
Medieval philosopher John Locke and many others have shown interest in signs and the way they communicate. 

The etymology of the term semiotics is form Greek word sema- “marks, signs” (singular semeion). It is defined as the 
science of doctrine of signs. The English philosopher John Locke (1696) first used Greek word ‘seemeiootikee’, in the 
modern sense. The word semeiotica is still used in Italy to refer to the study of symptoms in medical science. 

Modern semiotics emerged through the work of two linguistics theorists: Swiss Linguist Ferdinand de 
Saussure (1857-1913) and American philosopher Charles Saunders Peirce (1839-1914). These two philosophers 
inspired Charles Morris, Thomas Sebeok, Umberto Eco, and Roland Barthes. 

Ferdinand De Saussure published Course in General Linguistics in 1915. For Saussure, a sign has two 
components, the signifier or "sound-image", and the signified, or "concept". The relationship between signifier and 
the signified is arbitrary. He used the term “semiology” to refer to the systematic study of signs. Nowadays, the term 
“semiotics” is being used. 

The most widely used definition of semiotics comes from Peirce who is considered to be the co-founder of 
the semiotics along with Saussure. He defines semiotics as the “doctrine” of signs “stands to somebody for 
something in some respect or capacity.” (Peirce, 1958) 

Although semiotics is both a sphere of inquiry and a meta-analytic tool that has been used in philosophy, 
anthropology, sociology and linguistics, examination of signs in an educational context is a relatively recent 
phenomenon. (Cassidy, 1982) The field of semiotics became popular since it has been used in theatre, medicine, 
architecture, puppetry, television, tourism, and now on the Internet. It includes study of how animals communicate 
(zoo semiotics), of nonverbal communication (kinesics), of aesthetics, of rhetoric, of visual communication, of myths 
and narratives, of anything that allows us to make meaning and sense of the world. (Danesi, 1994) 

Semiotics looks at how the meaning is generated in "texts" (films, television programs, fashion, foods, etc.). 
The meal with steak, mashed potato, and apple-pie conveys meanings beyond the food. It shows status, taste, 
sophistication, nationality, and so on. (Berger, 1982) 

In his influencial book, A Theory of Semiotics, Umberto Eco (1976) defines semiotics as “the discipline 
studying everything, which can be used in order to lie.” Eco continues, “Semiotics is concerned with everything that 
can be taken as a sign. A sign is everything which can be taken as significantly substituting for something else." 

(Eco, 1976) 

Culture and Semiotics 

Human beings talk, write, blink, wave, and disguise themselves. They put up signposts and erect barriers to 
communicate messages to other people. They produce and interpret signs. But even if no one intends to 
communicate anything, sign processes are taking place: A doctor interprets the symptoms of a disease, a dog follows 
a trail, and a thief triggers an alarm. Then, what is not a sign? Almost every action, object, or image mean something 
to someone somewhere or sometime. From our gestures to what color dress we wear is a sign that has meaning 
beyond the object itself. 
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Construction ofMeaninu 




Figure: Construction of Meaning 

The meaning of signs or representations is dependent on social, cultural, and historical contexts. We 
construct meaning based on the physical appearance of the sign; our previous personal and cultural experience; time 
or era we live in; and context or place it occurs (See the figure above). There is not one meaning or interpretation of 
an each sign. There are multiple sides and points of view to each sign. 

As in James Mangan (1981) doctoral thesis. Learning through pictures provides interesting examples to 
illustrate both cultural and cognitive limitations to the ability to understand pictures. These limitations must be taken 
into account when designing learning materials not only for rural villagers in Dr. Mangan's study, but the global 
villagers in the world. Mangan says “Cultural differences in perception is more subtle and numerous than most 
educators suspect.” 

Looking at three different pictures of bears from three different cultures may be a great example. One is a 
picture of Yogi Bear from Western cartoon that can be seen as a dog if one has never seen the cartoon before. 
Another bear picture from North American Natives, a picture of a Tsimshian bear does not look like a bear to me. Or a 
hand drawn picture of a bear, may just be perceived as many lines for someone who has never learned to see the bear 
picture that way (Mangan, 1981). 

The greatest difficulty for international language of signs is that the same denoted sign can have many 
different connotations. When messages are attempted across cultures -whether based on age, economics, gender, 
ethnic background, location, it is decoded differently. For example, in many cultures eye contact between two 
individuals talking to each other is a sign of interest. In other cultures, it may indicate disrespect, insult. One of my 
advisees had a trouble with a few teachers who asked me why my student never participates in class. As a Korean 
student, we found out when she was ready to answer. She bowed her head to indicate she was ready for an answer 
while the teacher assuming she was avoiding the eye contact. 

As Lester indicates, “Humans always see and hear through the filter of who they are within a community.” 
Consequently, the meaning behind any sign must be learned. In other words, for something to be a sign, the viewer 
must understand its meaning. If you do not understand the meaning behind the orange color of a jacket, it isn't a sign 
for you. (Lester, 1995) 

W. V. Quine’s famous example shows the impossibility of learning a language from scratch. 

Imagine yourself as an adventurous linguist marooned in some unknown territory hosted by a tribe. You are 
trying to learn their language, and at some point your host points his index towards a rabbit running in the 
distance and says “gavagai”, what could you deduce from that? That “gavagai” means rabbit? Or “in the 
distance”? Or “brown-grey”? Or “this is our meal for tonight”? Or that this rabbit is called “gavagai”? Or 
that it means, “Go home!” If the indexical sign is not clearly identified, (and how could it be identified?), this 
iconic way of learning a language (word meanings by analogy with existing things) cannot work. (Codognet, 
1990) 

Every single thing has meaning and gives a different message depending on where it is located and who 
sees it. For instance the color “red” implies different things. Red means stop when it is on a traffic light, stands for 
blood in medicine. If a woman wears a red dress, or a man wears a red armband, it means something different. In 
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general, red in western culture means usually danger, hot, sexy, embarrassment, left wing or radical. Whereas red 
brings different things in my mind since I grew up in Turkey. Traditionally, red was the color of the wedding gown. 
Still in some villages it is possible to see red gowns. Also, during a Festival, the blood of a sacrificed animal can be 
put on the face for religious purposes. That's why the physical property of red light stands to somebody (a motorist, 
a pedestrian, a political demonstrator, or a Turkish woman, etc.), for something in some respect or capacity (stop, 
political perspective, wedding). (Danesi, 1994) 

Semiotics in Media and Communication 

As Albertson states, just as the introduction of sound into the medium of film brought change in theatrical 
form, so has the introduction of Internet communication into human discourse influenced semiotic forms throughout 
world cultures. With the introduction and penetration of new media and computer mediated communication systems, 
human interactions are changing at rapid speed. This change could possibly be more rapid, "than any other time in 
history" and instant communication with people all round the world changes the way we perceive space and 
time. (Albertson, 1997) 

Human beings created a world of messages and meanings and continue to create new ones to look for the 
meaning of life. In order to communicate with each other and leave their stories for the new generation, humans have 
been using the power of images and symbols since the beginning of the human history. From stone surfaces to 
animal leathers, from paper to Internet, "the story of humankind has been written, is being written, and will be written 
upon its virtual walls (Albertson, 1997)." The electronic images now being shared all round the world and the 
necessity to understand and study these signs are becoming more important. 

Philippe Codognet (1990) perceives Semiotics as a “... powerful tool in order to rephrase information theory 
and computer science and shed a new light on this global phenomenon.” In his Semiotics of the Web article, 

Codognet looks at the traces of signs in the history and talks about the importance of images, sculptures, and 
pictures in churches in order to educate illiterate people from religion to geography. Especially in Western culture, 
the use of images to represent knowledge and synthesize information has a long background in the history of ideas 
(Codognet, 1990). Even after the invention of the printing press, the power of images, symbols, and icons stayed the 
same. 

Eirst Internet pages on the World Wide Web had more icons, and pictures. Today, some of the symbols that 
were created by the programmers stayed as an icon. However, trying to find an icon for every link faded away. Yan’s 
research brings various examples from library web pages. Eor instance, to signify the library catalog, a button with a 
picture of a book now just turned into a link with a written library catalog link. (Yan, 1995) As Codognet said “pure 
iconography is not possible, as icons have to present themselves as such, to display their own icon-ness.” 

Being limited to ASCII (The American Standard Code for Information Interchange) texts and the ejqransion 
of the Internet communication, early Internet users developed a series of nonverbal symbols and signs. Such signs 
could signal play, emphasis, exaggeration, or shouting. The signs were informal and intended to facilitate 
communication. These signs are part of the construct, which became known as 'netiquette' (Glister, 1993). Eor 
example, : - ) © used to describe the happy mood of the author, and abbreviations such as POS to inform the instant 
message partner “Parent is Over Shoulder”. 

Although many words in the world of computing sound friendly such as mice, fields, bugs, turtles and they 
have connotation of the natural world, or some from everyday life queue, address, bus, network, and the audience 
still will face many obstacles in viewing the web pages. Just being able to read the written text may not be enough to 
decipher the message on a web page. The pictures, animation, video, and hyperlinks need to be designed for 
international community. The audience may feel, unjustifiably, that difficulties in use result from their own failure to 
understand rather than from the failure of the computer specialist or an instructional designer to communicate 
effectively (Gerver, 1986). 

Media Literacy 

The definition of literacy has been changing rapidly. As Ernest L. Boyer said “It is no longer enough to 
simply read and write. Students must also become literate in the understanding of visual images. Our children must 
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learn how to spot a stereotype, isolate a social cliche and distinguish facts from propaganda, analysis from banter, 
important news from coverage.” 

Media Literacy was defined at the Aspen Institute in 1989 as “ability to access, analyze, communicate, and 
produce media in a variety of forms.” Media literacy is more than asking students to simply decode information that 
they experience in the media, but they must be able to talk back and produce media. 

Today, gaining Media Literacy skills is becoming more important. Preventing a new generation to see TV or 
use Internet is not going to solve the violence, hunger, or world problems. The more we learn about TV, use the 
Internet, the more likely we see and understand the world. As Len Masterman (1985) says, “Media are symbolic 
systems; not simply reflection of reality which must be accepted, but with languages which need to be actively read, 
and interrogated.” 

Stories like Zack written by Alan November in his article entitled, “The Web: Teaching Zack to Think” gives 
an example of a student who wrote a history paper about how the Holocaust never happened. Zack sited his resource 
from a university professor’s web page. Zack story is a great example of the importance of new media literacy skills. 
“Just because it’s on the Net does not make it true.” (November, 2001) Students need to be able to identify who the 
author, how it is being presented, and whose point of view is being presented. 

Glister defines Digital literacy as: 

Ability to understand and use information in multiple formats from a wide range of sources when it is 
presented via computers... (Not) only must you acquire the skill of finding things, you must also acquire the 
ability to use those things in your life. Acquiring digital literacy for Internet use involves mastering a set of 
core competencies. The most essential of these is the ability to make informed judgments about what you 
find on-line. (Glister, 1997) 

Cornellia Brunner and William Tally (1999) places emphasis on the importance of effective research skills in 
education and encourges educators to incorportate new media literacy skills into the curriculum. In addition to 
reading, writing, and aritmetic. Rich Thome (1996) adds another R (research) in his article. Fourth R is Research. In 
the information and communcation age, even though reaching information on the internet seems to be in our finger 
tips, finding a reliable resource and ability to make informed judgments about what one finds on-line requires new 
skills." (Glister, 1997) 

Semiotics and Literacy 

Semiotics complements and promotes the ideals of Media Literacy education. Semiotics is one of the 
approaches to media education and new media literacy. It opens a new way to the study of media literacy. We can 
not only study the alphabet of deaf-mutes, symbolic rites, polite formulas, military signals, and religious symbols but 
also study the commercials, sit -corns, soap operas, and bulletin boards. In a sense, we all are using, deconstructing, 
learning, and reading signs even though we never studied Semiotics. We learn new signs everyday as the signs 
evolves continually in their meaning or significance all the time. 

The aim is to develop awareness about print and the newer technologies of communications so that we can 
orchestrate them, .... And get the best out of each in the educational process. Without understanding of media 
languages and grammars, we cannot hope to achieve a contemporary awareness of the world in which we live. 
(McLuhan, 1967) 



Both Carmen Luke (1994) and Marshall McLuhan (1967) talk about “grammar” and “language” of moving 
images. Musicians use notes and scales to communicate, dancers use movement, and scientists use mathematical 
notations. Media producers also have a unique system of grammar and language, and learning to read it in all its 
complexity is cmcial to being media literate (Scott & Yildiz, 1996). 

Film, TV, commercials, and the Internet are texts and have their own language and grammar. Language is a 
social institution that gives us the rules and conventions and speaking is the action part that is based on language. 

Education and Signification 



220 




Kimberly J. Sloan (1995) wrote a reflection piece on her “teachable moment.’’ While she was working at a 
juvenile prison, she drew a picture of a crown to represent Prince Hamlet of Denmark. It was interpreted as a gang 
symbol by the class. By drawing the crown, she had somehow affiliated herself with the Ghetto Boys of Indianapolis. 
This angered her students who consider themselves members of a rival gang. She used the drawing of the crown on 
the board to be interpreted as an icon. She wanted to illustrate the power structure of Shakespeare's play Hamlet by 
mapping the character names on the board to show their relationships in the play. However, some students did not 
accept the drawn crown as an icon, but instead perceived it as a symbol. To them, the crown was a gang symbol that 
demonstrated their affiliation to that particular gang. 

While I was working as a media specialist in a high school, I had an interesting incident. Teachers were 
required to save students' progress reports on a floppy disk, and then upload the file to the registrar’s office though 
a school wide information system. One day, one of the teachers told me that the registrar received her file without any 
data. She was sure she followed all the directions in the manual and even the computer said “OK.’’ Except she forgot 
to upload her file since she saw "OK" next to her name, and the date it was sent. It took us a while to find out which 
step she missed and why she thought she was right. When she saw "OK" (zero kilobytes), she thought it was "OK" 
meaning the message transmitted safely to the registrar’s office. 

When I attended a presentation given by a dance teacher Luana in National Association of Multicultural 
Education conference in the fall of 1996. Luana talked about the differences in art and dance among cultures, and 
gave interesting examples, even encouraged us to dance and present ourselves through dance. Through dance and 
movement, humans define themselves. In western culture, the dance especially “bale’’ includes upward movements. 
The dancers are as if trying to reach to the sky. In Africa, the dancers dance toward the earth. Eor instance, whirling 
dervishes (Sufi) try to reach to the sky (God) with one hand and the other hand points to the earth. The movement in 
dance reflects cultural differences. As one teacher pointed out, African descent students walk as if they are reaching 
to the ground. Some may interpret this as an attitude towards a system or teacher, but as the speaker points out this 
is just a movement coming from the student's culture. 

The teachers’ role in education is critical and important. Eor Paulo Ereire (1993), “Education must begin with 
the solution of the teacher-student contradiction, by reconciling the poles of the contradiction so that both are 
simultaneously teachers and students.’’ Although it is almost impossible to understand each and every students’ 
background and culture and their interpretation of the signs and symbols, as Sloan (1995) suggests “We can use our 
various interpretations of signs as a starting place for discussion of our often opposing value systems, to create 
interesting juxtapositions, and to investigate others "personal structures" to broaden our own experiences.’’ 

In the millennium, we are going to be surrendered by more and more images from bulletin boards to Internet, 
from advertisements and banners to book covers. The Internet and new technologies creates new images, icons, 
symbols, and metaphors to study for us to make sense of the Cyberspace. 

The mass media . . . play a critical role in the social construction of knowledge concerning race and ethnicity 
. . . Eor the full flourishing of multicultural education, scholars need to develop more sophisticated ways to 
explore and assess media-based multicultural knowledge construction. (Cortes, 1995) 

Since new generation is spending more time with the mass media, their interpretation of the world is mainly 
based on images they have seen. As writer Walter Lippmann says, “Whether right or wrong ... imagination is shaped 
by the pictures seen ... Consequently, they can lead to stereotypes that are hard to shake.’’ 

Instead of consumers of media, new generations need to be the producers of media. The more they study 
and experience media, the more likely they see, understand, and interpret media. Len Masterman (1985) considers 
media literacy education is a crucial step towards “participatory democracy.’’ Masterman adds, “Media Education is 
both essential to the exercising of our democratic rights and a necessary safeguard against the worst excesses of 
media manipulation for political purposes.’’ 
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Abstract 

Constructivism, or more precisely constructivist theory talks about the authenticating the learning environments. 

In order to create the more constructivist and authentic learning environments, designers should fallow the certain 
characteristics that are given by Jonassen (2002 ) on his web site 

(http://www.coe.missouri.edu/~jonassen/courses/CLE). The purpose of this study is to reveal novice software 
designers ’ problems within which they are in the process of constructivist computer software design. Qualitative 
research method will be used to collect and analyze data for this study. Also, this study will proposes 
recommendations on how to improve the training of novice educational software designers. 

Within educational software design, the term constructivism has taken a very important role ever since 
constructivism became so popular at the beginning of 1990s. Educational technologists and instructional designers 
have developed many constructivist computer based programs or assimilated constructivist strategies into 
educational programs . However, despite the continued emphasis on constructivism, good examples of 
constructivist principles applied to software development are rarely found in programs in Turkey. Perhaps there is a 
lack of theoretical knowledge among software designers or they may be having some problems during the 
development process. 

Constructivist approach to instruction is different from those traditional methods, so classical methods of 
instructional design use some inappropriate design tools to create constructivist learning environments. In the heart 
of constructivism the students situates the learning experience within his or her own experience and that the goal of 
instruction is not to teach information but to create situations so that the students can interpret information for their 
own understanding (Heinich, Molenda, Russell, & Smaldino, 1999). 

Wilson (1996, p.5) defines a constructivist learning environment; a place where learners may work together 
and support each other as they use a variety of tools and information resources in their guided pursuit of learning 
goals and problem solving activities. This definition has no universal acceptance for constructivist learning 
environment but juts a starting point for this study. 

Method 

This study attempted to examine novice software designers’ problems when faced with the task of 
designing constructivist-based computer software programs. At Anadolu University in Turkey, the Division of 
Educational Communications and Planning offers a software design course called Learning with Technology. Every 
spring semester approximately 4-5 graduate students take this courses as an elective. At the end of the course, each 
student has to develop educational software to be used for instructional purposes in a variety of areas. Some 
programs are directed at the development of a particular institution’s or company’s professionals, or are meant to 
support formal school courses. The research project examines Jonassen’ s characteristics of meaningful learning to 
create constructivist learning environments (Jonassen, 2002, 

http://www.coe.missouri.edu/~ionassen/courses/CLE/index.html) . These constructs include addressing active, 
constructive, collaborative, intentional, complex, contextual, conversational, and reflective characteristics of 
meaningful learning to provide guidelines to students to help them create software containing these constructs in 
their own programs. After examining the guidelines each student developed his/her own software based on these 
guidelines and tried the software out to see if it matched the constructivist learning environments’ characteristics. 

Hoepfl states that (1997, http://scholar.lib.vt.edu/eiournals/JTE/v9nl/hoepfl.html ) the ability of qualitative 
data to more fully describe a phenomenon is an important consideration not only from the researcher’s perspective, 
but from the reader’s perspective as well. "If you want people to understand better than they otherwise might, 
provide them information in the form in which they usually experience it" ( Lincoln and Cuba, 1985, p. 120 ). Qualitative 
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research reports, typically rich with detail and insights into participants’ experiences of the world, "may be 
epistemologically in harmony with the reader’s experience" I Stake, 1978. p. 5 1 and thus more meaningful. According 
to those explanations of qualitative research, the researcher used this method to gather rich and detailed data for this 
study. 

4 graduate students in the course participated in the study. Interviews with the students, and the online 
discussion group is being used to gather data for this study. Interviews with the students will be all sections of the 
course such as in the beginning, middle, and the end. In the beginning the instructor and the students discuss the 
guidelines and converse about their usage for designing instructional software. After beginning the developing 
process the conversation continues between the instructor and students. At the end of the course the in depth 
dialogues are analyzed using a content analysis technique. All of the conversation process with the students are 
collected and coded for analysis. 

The second data collection tool is analysis of an online discussion group. The online discussion group 
consists of the students who are taken the course and willing to participate to the study. All conversations, chats, 
and e-mails among them are saved to a computer with the intention that they could be used for elaborative data for 
the research. 

Analysis of the data begins with identification of the themes emerging from the raw data. During this 
coding period the researcher is identifying and naming the conceptual categories into which the phenomena 
observed will be grouped. As the raw data are broken down into manageable chunks, the researcher is coding the 
data and translating it into the story line. The results of this research indicate an in depth analysis of the strengths 
and pitfalls that students encounter when using these constructs in a prescribed setting to design constructivist 
learning environments in a computer software program. 

Novice designers chose statistics as a subject (content) to design a constructivist learning environment. 
Their target group (learners) was also graduate students who have not any knowledge and experience about the 
research statistics. 



Results and Discussion 

Active: In order for the learners’ activity, novice designers were embedded some cases into the program. 

The cases were designed with the collaboration of a statistics practitioner and a full statistics professor. The 
designer, practitioner and professor were worked together as a development team for the project. The most 
discrepancies for the team were developing the real life situations in the cases. 

Constructive: Learners integrate new ideas with prior knowledge in order to make sense or make meaning or 
reconcile a discrepancy, curiosity, or puzzlement. They construct their own meaning for the phenomena (Jonassen, 
2002). Knowing what the students knew was very painful process for the designers. 

Collaborative: Online discussion groups and electronic mail were the collaboration tools for the students. 
Also, collaborative cases were helped the designers to built collaboration among the students. 

Intentional: Learners were encouraged with highly motivated goals such as after completing this program 
you will be able to do your future research statistics by yourself before, during and after the program. It was mostly 
easy for the designer to indicate this goal for these level students. 

Complex: Creating ill structured and a complex learning environment was too difficult for the team. The 
reason for this all the members of the team were studied in the traditional system. In this system, all refined and 
oversimplified knowledge were given to them in their all school life. Therefore, they had difficulty to build a complex 
and ill structured learning environment. 

Contextual: A great deal of recent research has shown that learning tasks that are situated in some 
meaningful real world task or simulated in some case-based or problem based learning environment are not only 
better understood, but also are more consistently transferred to new situation (Jonassen, 2002). In the cases real life 
contexts were developed by the designers. 

Conversational: In order to socialize the learners online discussion groups and electronic mail were used 
in the program. Also the cases were designed to make them collaborate with each other. 

Reflective: Cases and examples were connected to another situation with which learners can articulate what 
they are doing, the decision they make, the strategies they use, and the answers that they found. 
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Designing a constructivist learning environment seems to be too hard for novice designers. Creating a 
good designing team, and conversation in the all designing process with the novice designers could helped them for 
better understanding to design and develop a constructivist learning environment. 
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Abstract 

This paper outlines the efforts of a departmental student organization in the Instructional Systems Technology 
department at Indiana University. This organization is named Graduates in 1ST ( GIST). As officers during the 
2001 -2002 academic year, we focused on addressing academic, professional, and social needs, our work in the 
department was guided more by the traditions of the department and our own sense of what activities were 
necessary and valuable for the students in our department — our colleagues than by any systematic attempt at 
creating a sense of community. A brief glimpse of the past is offered to ground the description of the activities that 
were offered by the student organization in 2001-2002. After these activities are described, we share a bit about 
current and future directions for the organization. The paper concludes with a discussion of the difficulties of 
creating a sense of community and shared culture in a graduate department. 

Past 

There has been a graduate student organization in the 1ST department at I.U. even before there was an 1ST 
department. The student organization began when the Department of Audio-Visual Instruction was in its heyday at 

I.U. Professors who have been here for some time agree that the student organization has been around for over 30 
years although there is little record of activities or accomplishments of the early years of the organization. 

According to one professor, the student organization has ebbed and flowed over the course of the years. 
Within periods of activity characterized by either recession or advance there was still more fluctuation that seemed to 
be at least partially explained by changes in leadership of the organization. As GIST is currently organized, there is a 
two-level membership structure in the organization. All 1ST students are members; faculty and students can become 
active members by paying yearly dues. The organization is led by six cabinet-level positions: president, vice- 
president for professional development, vice president for social activities, vice-president for information services, 
treasurer, and secretary. 

When the 200 1 -2002 academic year began, there were no officers since elections had not been held the 
previous spring. Two students formed a nominating committee and soon candidates for cabinet level positions were 
nominated. After the elections in mid -September, the GIST executive cabinet took up weekly meetings at the end of 
September 2002. We were excited to carry on the traditions of the department and to forge ahead in new directions. 

Present 

The description of our organization which follows is organized around four main ideas. 

1. GIST as a voice for students 

2. GIST as a social organization 

3. GIST as a support for professional growth 

4. GIST supported by technology 
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GIST As A Voice For Students 



One of the responsibilities of the organization is to be a voice for students. Fulfilling this responsibility has 
taken a number of forms from advocating increased support for technology learning, to attending faculty meetings on 
a regular basis, to keeping thorough records of executive meeting minutes. 

Advocating Technology Support: Students coming to our program span a range of technology capabilities. 
The core classes (a group of classes required for the minor. Master’s and Ph.D. tracks) are particularly demanding of 
these skills. The department has experimented with a number of ways of addressing this issue. During the first 
semester of 2001-2002 the GIST cabinet discussed the possibility of co-sponsoring technology workshops for first 
year students. We were reluctant to offer too many since we did not feel that it was necessarily our responsibility to 
train new students on technology and yet saw the need for skill training since all of us had just experienced the 
demands of the spring production class. We approached the department with a plan in which GIST would sponsor 
one workshop for every workshop offered by the department. The faculty supported the plan and a number of 
technology workshops were organized by one of the GIST officers. All of the workshops were directed at helping 
students achieve success in the spring production class. GIST made the following recommendations to ensure the 
quality of the workshops and that the workshops could be a stepping stone to a long-term solution to the question 
of supporting student technology learning: 

1. Each workshop should include a high-quality visual aid for those attending. Basic guidelines should be 
established for these handouts and submitted for review prior to the session. The department should 
cover copying fees for the handout. A copy of the handout should stay on file with the department. 

2. An evaluation crafted jointly between GIST and the department should be formulated for all workshops 
offered as part of this initiative. 

While various workshops were offered and well-received, they were not evaluated in any systematic way 
and thus little was learned to guide in future technology training decisions within the department. The current GIST 
executive committee intends to follow-up on this concern for the coming year. 

Faculty Meeting Minutes: GIST has been attending the faculty meetings in the Instructional Systems 
Technology Department at Indiana University since November 02, 2002. Although provision for student 
representation in departmental meetings had long been part of the GIST charter, a regular student presence in faculty 
meetings had not been established for some time. The following criteria were set by the GIST executive committee to 
re-establish GIST’s participation in the faculty meetings: 

1. In re-establishing this relationship it was decided that two members of the GIST executive committee 
would attend all faculty meetings. 

2. These student attendees have the opportunity to bring issues to the faculty meeting by submitting 
items to the department chair for the meeting agenda in advance of the meeting. 

3. The student representatives compile minutes of the meeting for the purpose of sharing them with 
students. 

4. Prior to being posted on the GIST web site, the notes from these meetings are approved by the 
department chair, in order to ensure the flow of accurate information. 

These minutes are designed to help students get a picture of the workings of the department and to 
generate thought, discussion and idea sharing among students. Included in the minutes are possible implications for 
students. These implications are the interpretations of GIST representatives. 

We have been warmly received by the faculty in each of the faculty meetings. At times the faculty solicits 
our opinions, but most often we listen and observe. 

GIST Executive Committee Minutes: In order to truly represent students — to be their voice, it was important 
to strive for transparency. One way that we attempted to accomplish this was through making all of our meeting 
minutes public. We are not aware of how or if these minutes are used by other students but we do know that they are 
quite valuable to us since they provide another layer of organization to keep us focused and as we mention later, are 
part of a valuable documentation of organizational processes. 
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GIST as a Social Organization 

One main goal of GIST is to help 1ST students feel part of a community and to celebrate each victory and 
success despite the many challenges of being a graduate student. In an effort to address social needs of the 
department a number of events were sponsored including, Hallowine, 1ST lab snacks at Christmas, Portfolio 
celebrations, and 1ST Follies. Those activities were developed as an effort to develop this sense of collective identity 
and to offer a supportive environment to all members of the 1ST community. 

Social functions tend to be associated with fellowship, eating, drinking, and enjoying the company of each 
other but it is more than that. These times are about taking time off, sharing life’s moments, building relationships, 
and creating a community that cares about its members. 

Hallowine: The Halloween party (known among ISTers as Hallowine) was celebrated on the weekend before 
Halloween. 1ST students, faculty, families, and significant others were invited to wear costumes and to enjoy a 
potluck dinner in a Halloween atmosphere. Sixty to eighty people attended the event that took place in the home of a 
faculty member. Even though Halloween is a traditional American holiday a considerable number of the attendees 
were international students. 

Portfolio Celebration: The portfolio review is done at the end of fall and spring semesters and is the 
celebration of an important milestone for Master’s and Ph.D. students. The portfolio review is a perfect opportunity 
to bring together the academic, professional, and social facets of 1ST life for all 1ST students including first year 
students who historically were not part of this celebration time. Fifty to sixty people attended the event including 
faculty, new and older students along with a considerable percentage of students who had submitted the portfolio. 
Faculty attendance was almost one hundred percent. 

1ST Follies: 1ST Follies is a cabaret type theatrical event held late in the spring semester every year since 
1996. This year the program consisted of musical and other artistic performances (music and dances from Thailand, 
China, Turkey, and Mexico, which reflected departmental diversity) and humorous skits, live and recorded, produced 
by students and faculty. As the name implies, the overall purpose was to celebrate the talents and follies of the 
department over the past year. In the School of Education auditorium, eighty to hundred people attended the event 
including students, faculty, and their families. A buffet was served after the show. No admission was charged. 1ST 
Follies was an event produced by all the community for family and friends ! 

GIST As A Ssupport For Professional Growth 

Round Talks: Round Talks are held semi -regularly and hosted by someone held in esteem for their expertise 
in a specific area of concern to 1ST students. The expert, however, is not designated as the “sage on the stage’’ and 
functions as a facilitator of discussion. These gatherings were originally conceived to provide opportunities for 
interactions not always available in the classroom and in an atmosphere less formal than an instructional setting are 
important contributions of the Round Talks. Five Round Talks (Technology Toastmasters, What it Means to be a 
Designer, Demystifying Portfolios, Advice for New Students, and Getting Started With EndNote) have been hosted 
since their beginning in the 2001-2002 academic year. Three more Round Talks are being planned for January, 
February, and March 2003. A short evaluation concludes each Round Talk. 

It seems evident from early reactions that venues such as Round Talks are one way of addressing a variety 
of student needs. There is a need for these events to be better promoted so that more students can engage and 
participate. Some of the topics suggested for future were: publishing, consulting practices, resume design, qualifying 
exams, the process of conducting research. 

1ST Conference: This annual conference serves as a forum for the discussion and exchange of information 
on the research and development on all topics related to Instructional Systems Technology. This is an excellent 
opportunity to practice presentation skills, to build a portfolio, to become aware of special interests represented in 
the department, and to extend collegial interactions between students and staff beyond the confines of the 
classroom. This conference opportunity is not only for presenting ideas but also for generating discussion, 
conversation and discourse, thereby seeking ways to enhance interaction and community feeling among the 1ST 
Community. 

The conference for 2002 year was scheduled for March 22. Up from five presentations in the first conference 
(2000-2001), there were 22 presentations given by individuals or by groups. There were some people that participated 
in more than one presentation. 
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Topics included: 



• 3-D environments for children 

• Lessons from an Online Dis tance Masters Program 

• Designing Digital Young- Adult Fiction 

• Instructional Theory 

• Cryptography 

• Peer Support in Graduate Courses 

The GIST committee intended to provide feedback on the subject of both the conference organization 
(conference evaluations) and individual presenter feedback (presenter evaluation). Although we had high rates of 
returns in presenter evaluations we have not had any feedback about the organization of conference. 

The department chair, Professor Elizabeth Boling, opened the conference and the dean of the school of 
Education, Gerardo Gonzalez, gave a brief welcome. The keynote speaker was C. Thomas Mitchell who presented on 
User-Responsive Design: Reducing the Risk of Eailure. Eaculty compliments regarding the 1ST conference 
organization following the conference indicated that the department was in need of and expected such academic 
events that bring together the 1ST Community. We hope that the conference can continue to grow and that 
eventually it will attract presentations from departments at I.U. with interests that intersect with those of our 
department and that eventually the 1ST conference at I.U. will attract the attention and attendance of other 
universities. 

GIST Supported By Technology 

Technology has played a major role in supporting our efforts as a student organization. Specifically, the 
GIST web site has been re-organized and structures are in place to provide for more reliable knowledge management 
for future leaders of GIST and for GIST members. 

GIST Web Site: At the beginning of our service as GIST officers, our web site was one of the main concerns. 
In the past, the website was just a personal file space and it had no significant meaning to the 1ST Community. It was 
used as a storage bin and a file sharing portal. We wanted something more for the site — to create a sense of student 
ownership — a place that would allow our members, and eventually all 1ST Community, to see the big picture of 1ST. 

The web site we inherited consisted of a home page, mission page, officers page, photo gallery, and the 
GIST application form. The Homepage was just an entrance where all the links to the different parts of the website 
could be found. Mission page gave the background information and statement of GIST’s objectives. The officers 
page gave the contact information of past officers, and described their duties. The photo gallery was a repository of 
pictures from past events. Einally, the GIST Application Eorm was a pdf file that could be printed and filled by people 
who are planning to become GIST members. 

Some initial reorganization of the web site took place almost immediately including the use of standard fonts 
and colors, the organization of existing files into directories, and the addition of a page titled “Library” to archive 
documents and pictures. Even with these changes, more work was needed. 

Web Site Redesign: The GIST web site was serving a very narrow focus of archiving pictures and 
documents with little thought to their organization. The web site offered little in the way of addressing the broader 
mission of GIST to provide for the social, academic, and professional needs of 1ST students. In order to address this, 
a web team was formed from within the GIST membership. This team was charged with analyzing the present needs of 
students with regard to the GIST web site and with designing a web site to address those needs. A team of analysts, 
designers, and project managers was formed to develop a survey and to implement the findings of that survey in a 
redesign of the web site — a good opportunity to apply the skills learned in the classroom. 

A web-based survey was developed to assess what students in the department wanted to see on the GIST 
web site. Since the web site analysis results were returned at the same time as our planning for the 1ST conference, 
there was no time to implement the thoughtful responses and suggestions that we did receive. The current GIST 
executive committee is considering a redesign of the GIST web site based on the feedback from that survey. 
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Knowledge Management for GIST: Originally, we decided to keep all of our documents from our meetings 
on the web in an effort to be a transparent and open organization. A Library page was created as mentioned above. 
Some of the examples of documents in the library include 1ST Conference documents (call for participation, review 
guidelines, conference schedule, etc.), meeting minutes (executive committee meetings, public meetings, and faculty 
meetings), and GIST documents from several social activities (Hallowine report, portfolio celebration report, 
logo/motto contest, etc.). 

In hind sight, the decision to create a virtual library has proven to be an important decision, not necessarily 
because students are accessing the meeting minutes or frequently visiting the web site, but because it is an effective 
way of managing the organizational knowledge that is accumulated by each group of GIST executives. We soon 
realized that our efforts to be transparent were resulting in the preservation of some departmental history. To date, no 
sustained effort has been made to document the history of the student organization in the department and what 
records there are exist in somewhat loosely organized binders. 

Another important role that the GIST web site will fill is that it will be a way to pass accumulated knowledge 
about how to organize and operate a student organization. This is particularly challenging since the leadership cycles 
in this graduate student organization are often only one year, possibly two. Without systematic efforts at preserving 
the processes that have been found to be useful, each executive committee has to begin at the ground level rather 
than build on the work of previous cabinets. 

Future 

The 2001-2002 academic year ended with elections to the GIST executive committee with only one officer 
seeking re-election. The new officers met at the beginning of summer to get to know one another and to briefly 
discuss directions for GIST in the coming academic year. The current cabinet is sustaining the innovation begun in 
2001-2002. New directions include a monthly e-Newsletter published by GIST and a conference planning position that 
has recently been filled by a capable candidate (suggested by GIST and funded by the department). 

Questions remain after all the efforts of 2001-2002. What is the effect of the student organization on the 
department? Is it possible that such an organization can contribute to a sense of community? Why is it so difficult to 
attract students to be active members to the organization? The 2002-2003 GIST cabinet has designed an impact study 
to probe student thinking about GIST membership and participation in GIST sponsored events. We hope that this 
will bring some insight into how to make the organization, and ultimately our department, better. 

Recently a faculty memb er familiar with GIST history commented that the idea of community is connected to 
a sense of professional identity and that few programs in higher education offer examples of strong, culture -building, 
experiences. As counter examples the senior member of the faculty offered the experiences of Army and Navy 
academies where ascension through the ranks creates a strong bond and sense of identity among students. In these 
environments, professional identity is connected to the activities and experiences of the academic setting. 

There are certainly challenges to the development of community in a graduate department. The ranks are 
there. Master’s and Ph.D. tracks, but progression from one rank to another is largely an individual affair. The short 
time that Master’s students are in the department makes it particularly difficult to establish a sense of belonging and 
identification with others as professionals in the sense of community building as described by Lave and Wenger 
(1991). The challenges for creating community among doctoral students are also many. While their departmental 
tenure is certainly longer than the Master’s students, the intense specialization and program focus along with 
teaching and family duties may limit if not prevent participation in voluntary departmental events. 

Using the logic of the professor mentioned above, it seems that building community in a graduate 
department, if it can be done, would be best achieved by focusing on the professional development of students. This 
might be done in a variety of ways but could certainly be helped by accentuating the milestones and rites of passage 
that students do experience as parts of their respective programs. In our department this might require a greater 
emphasis on celebration and support of core classes, on portfolio celebration, on passing the qualifying exam or on 
getting a dissertation proposal approved. 

Another real challenge to creating a sense of community is the difficulty of passing along understandings 
and processes — in a word knowledge management. Because the transient nature of being a student affects GIST 
officers and members alike, much knowledge is lost in transitions between cabinets. One real boost to this effort has 
been the commitment of the student organization over the past two years to maintain thorough records of what has 
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been done. The archives available on the web site require constant maintenance and upkeep but are probably one of 
the supports in place for preserving knowledge and processes that have come by hard work. 

In the end, community is a matter of degree. There are strong and week communities. The 1ST community 
falls somewhere on that weak to strong continuum and we are reasonably sure that for some people, at least, the 
GIST activities and efforts are one of a number of factors that contribute to a sense of connectedness among 
students. We look forward to continuing our community building efforts in the challenging environment of the 1ST 
department at I.U. We look forward to addressing the challenges of building the professional identities of very busy 
and transient students and to developing processes for building departmental culture that can be passed to new 
generations of students. 
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Impact of P3T3 on Faculty Use of Web Course Tool 

Hua Bai 

Suttichai Chuvessiripom 
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Abstract 

This study explored the impact of P3T3, Purdue’s PT3 implementation grant project, on faculty’s use of a 
web course tool, WebCT. Quantitative and qualitative data were collected through online survey and in-depth 
interviews respectively. Results suggested that WebCT was convenient for faculty to put course materials online 
and conduct online communications with the students. P3T3’s service was effective in training and supporting 
faculty in using WebCT, through providing workshops, drop-in session and one-on-one assistance. 

Introduction 

The rapid development of information technology has made computers and computer-related technology an 
integral part of teaching and learning. According to Glenn (1997), computers have advanced from simple machines 
with limited functions and capabilities to powerful machines with sophisticated applications and high-speed 
networking capabilities. Since the mid-1970’s, schools and colleges have raced to keep up with the rapid growth and 
change of technologies. The main goal has been to prepare educators to be able to integrate technology into their 
classrooms effectively. The implementation of information technology has been closely linked to the evolution of 
faculty development (Shapiro & Cartwright, 1998). While some faculty members still hesitate, many are making efforts 
to improve their knowledge in technology and plunging ahead to enhance their teaching using technology. 

Although educators are expected to have the knowledge and skills to be able to use technology in their 
instruction, the use of new technologies in the classroom for teaching and learning has been and is still limited. Brand 
(1998) pointed out that despite the increased access to computers and related technology, educators often experience 
difficulty in integrating technology into classroom teaching practice. The main reason is the issue of educators’ 
computer competencies; teachers often are inadequately prepared to use instructional technology. To achieve 
effective integration of technology into teaching practice, adequate training must be provided for the faculty. 

Quick (1999) conducted a case study using personal in-depth interviews to help faculty in a community 
college articulate their instructional needs related to using technology. The results indicated that the faculty wanted 
training classes that fit their time schedule and location. Personal tutoring and technical support were suggested. 
Qther researchers (Dusick, 1998; Dusick & Yildirim, 2000; Groves & Zemel, 2000; Matthew, Parker & Wilkinson, 1998) 
have also confirmed that training and support staff are significant factors in helping faculty to effectively use and 
integrate technology into the classroom. 

Although much literature addresses the importance of helping faculty’s use of technology through training 
or other forms of support, few studies have described in detail the impact of these supports on the development of 
the faculty’s use of technology. This paper reports on a study that was conducted at a midwestern university to 
explore how a PT3 implementation grant project affected the faculty’s development of the use of WebCT, a web 
based course development and management tool, in their teaching. This paper explores the faculty’s use of WebCT 
and their perceptions of the impact of P3T3 program on their use of this web course tool. The study adds to our 
growing knowledge of faculty development in the area of classroom integration, and also provides a valuable 
reference for the other similar projects involving faculty professional development. 

Background of the Project 

P3T3, Purdue Program for Preparing Tomorrow’s Teachers to use Technology, a PT3 implementation grant 
project, is designed to achieve two goals. One is to prepare pre-service teachers to demonstrate fundamental 
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technology competencies, using technology as a tool for teaching/leaming, personal productivity, communication, 
and reflection on their teaching. The second goal is to prepare teacher education faculty in Education, Science, and 
Liberal Arts to teach pre -service teachers in technology-rich environments, modeling approaches that future teachers 
should use themselves. All faculty in Purdue’s school of Education as well as selected colleagues in Science and 
Liberal Arts are participating. 

To achieve the second goal, P3T3 provides activities and extended support for faculty members. These 
include: 1) workshops to train faculty members on various technologies, 2) regular help sessions where faculty may 
consult with P3T3 graduate staff, 3) brown-bag TechieTalk sessions where faculty share their successful experiences 
and gain new knowledge, and 4) one-on-one personalized assistance. The workshops provide faculty development in 
the use of various technologies that can be incorporated into courses, such as PowerPoint, WebCT, graphic imaging, 
video editing, ErontPage, Dreamweaver, and Plash. The help sessions are available every week. Paculty member can 
drop in anytime during the help sessions to consult with the P3T3 graduate staff about classroom technology 
integration. Paculty can also make appointments for one-on-one assistance with P3T3 staff at any other times. 

The advantages of online learning drive more and more faculty members to deliver courses online or use the 
Web to support traditional courses. WebCT is a web based course development and management tool that allows 
faculty to construct and manage online courses, put materials online to supplement existing courses, create online 
communication environments, and track students’ performance electronically. It is the “standard” platform for web- 
based courses and course support at Purdue. The easy-to-use features of WebCT make it popular with faculty. To 
help faculty master the tool, P3T3 offers three workshops that teach, step-by-step, how to design a course using 
functions available in WebCT. In addition, the P3T3 staff provides assistance for those who have difficulty. 

Methodology 

To investigate the impact of the P3T3 project on faculty’s use of WebCT, both quantitative and qualitative 
data were collected from those who participated in the P3T3 project and who used WebCT in their teaching for more 
than one year. A faculty member and two graduate students in Educational Technology conducted the research. 

They worked together to design the study, develop the research instruments, and analyze the data. 

Data were collected from both an online survey and in-depth interviews. To obtain information about 
general characteristics of the faculty’s use of WebCT, an online survey was conducted on WebCT. Twenty-two 
faculty members from the School of Education participated in the online survey. The faculty’s thoughts and 
experience with the tool and their viewpoints about P3T3’s support were explored more extensively using in-depth 
interviews with 3 faculty members who had been through the WebCT workshops and who used WebCT in their 
courses. 

The survey consisted of 9 multiple choice items concerning P3T3 participation and the use of WebCT, such 
as “When did you participate in a P3T3 start-up workshop?”, “Which features of WebCT do you use? ”, and “Have 
you ever sought assistance of one of the P3T3 drop-in help session?”, and so on. Erequencies of responses to 
survey items were tallied. The interviews with the faculty members took place on campus at the convenience of each 
individual participant. Data from the interview included in-depth information about faculty’s experiences using 
WebCT and their perceptions of P3T3 work. The participants were asked questions such as “How does WebCT 
enhance your course(s)?”, “What problems have your encountered when using WebCT?”, “Can you cite specifics of 
how P3T3 assistance helped you?” Interviews were tape-recorded and then the content was transcribed. Interview 
data were combined together and then analyzed by question. The results of both survey and interviews were 
combined together to comprehensively examine the effect of P3T3’s work on the faculty’s use of WebCT. 

Results 



General characteristics of the use of WebCT 

Results showed that 77% of the participants had attended P3T3 workshops on WebCT. All participants 
incorporated WebCT into their teaching. Most (59%) used WebCT for basic content presentation such as putting 
course notes, syllabi, and assignments online. Many (41%) posted and organized students’ grades and used the 
quizzes/survey feature. WebCT e-mail (71%) and discussion fomms (82%) were among the most popular features. 
Lew faculty members had experimented with the WebCT live chat feature. Among those who attended the 
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workshops, 25% continued to seek help from the P3T3 drop in help sessions, and 63% had requested one-on-one 
assistance. Besides the help from P3T3 staff, several faculty members had sought assistance from friends and peers 
and from other technical support resources around the university. Overall, participants rated the services from P3T3 
high: 50% great, 25% good, 6% ok, and 19% no opinion. 

Experience with WebCT 

As course instructors, the participants considered WebCT a convenient tool for teaching. One faculty 
member who had students do practicum in a high school used to visit the school often to learn about how student 
were going there. Using WebCT, he can get student information via the postings. WebCT cut down the number of 
his visits to the school, “It saves me time, because in the past, I have had to make many trips to the school. And now, 
I make only a few”. In general, the participants liked WebCT because it helped them better organize course materials 
for students. One faculty member commented: 

[It allows us] to extend the instruction beyond the classroom. We’ve been able to put up 
articles that students can look at and read... outside the regular class. That’s been really 
helpful, really useful. 

All three participants mentioned that, because of the features available in WebCT, they could conveniently 
make all course materials available in one place. 

One participant felt that using WebCT took much more time than a regular class, “It does take a lot of time. It 
needs a lot of support if you going to be successful with it”. However, she expressed the opinion that she would 
definitely use this web course tool again. She ejqjected to develop an entirely online course. 

Although they had a positive experience with WebCT, the participants thought that the interface was not 
intuitive, “Everything is there, but it is not intuitive in terms of how to navigate, how to find way around it”. 

Perceptions on P3T3 work 

Generally speaking, the participants were satisfied with P3T3 work in introducing technology. As one 
faculty member stated: 

I think that’s an excellent program when the School of Education sent around survey to 
ask what we thought were the outstanding features of the whole school of Education, and 
I said the P3T3 was one. It has so many important aspects, number one is it insists, or 
certainly intends that everyone who is preparing teachers be able to use technology on at 
least a minimum level. 

The participants reflected that the P3T3 workshops were very helpful in getting started with WebCT, as one 
faculty member pointed out “those workshops made me less afraid to work with WebCT”. However, one felt that the 
workshops went too quickly. 

The participants were satisfied with the accessibility of the P3T3 graduate assistants. One faculty member 
commented that help sessions and one-on-one assistance were very helpful and he learned more from the personal 
help session than from the workshops. Another one who called himself as Technophobe reflected that with the 
assistance of the graduate staff, he became less afraid to experiment with the use of technology. Another faculty 
affirmed P3T3 work, but suggested the assistants be more knowledgeable in technology: “The more they learn, the 
more useful they will become”. 

As to how to better support faculty’s use of WebCT, one faculty member specifically talked about the 
importance of personal support in managing the materials when building a WebCT site for a class: 

I think it really in some cases most helpful to help faculty is to have somebody available 
for them to help them (faculty) with actual materials content development, not somebody 
just help you with questions. I think what to do most helpful for faculty is to have an 
assistant like, when I am working on WebCT materials, if I could have someone whom I 
could give some stuff and say “Could you put this for me on WebCT?” I do a lot of things 
by myself, but I think if I had somebody else who could have helped me, that would be 
very helpful. 
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Overall, the participants thought P3T3 had a positive impact on their use of WebCT and expected more 
support from P3T3 graduate staff. They regarded P3T3 as a good resource in introducing and facilitating their use of 
technology. 



Discussion and Implications 

The results of the study indicated that WebCT was a useful tool by faculty members, though the design of 
the software was not perfect. The interviewed participants used WebCT to support their regular classes, but the 
experience with WebCT helped them to envision a fully online course delivered through WebCT. At this point, they 
began to think more of how to integrate technology into teaching. 

For the beginners, the WebCT workshops need to be structured to accommodate their pace and progress. 
After the workshops and while the faculty members are learning to use WebCT to build their courses, more support, 
especially immediate support, should be provided. This will help to reduce the faculty’s frustration with new 
technology. Faculty expect knowledgeable staff assistance. To better support the faculty’s use of technology, the 
graduate staff should have enough knowledge and skills to provide quality assistance. Further, for those faculty 
members who need personal support in developing materials for a WebCT course, P3T3 graduate staff need to help 
with actual materials development instead of just answering technology questions. 

This study showed that P3T3, a PT3 implementation grant project, has been helpful in facilitating faculty to 
use WebCT in their instruction. The results confirm that training and support are important in technology integration. 
In addition, faculty’s technology level and individual needs should be taken into consideration when designing the 
training activities and providing support. 
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From Theories to Practices: Ten Guiding Principles for Integrating Technology in 

College Teaching 

Mei- Yau Shih 
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Abstract 

Faculty Developer, Administrator, IT professional, and Faculty members will have an understanding of the guiding 
principles for helping faculty integrating IT for teaching on a college campus, and how to create an ongoing 
dialogue between campus technology services providers and faculty, as well as among services providers. Each 
campus should identify the priorities for planning steps and implementation guidelines, which are shaped 
profoundly by the campus culture and politics. 

Introduction 

The rapid growth of information technology (IT) and pressures for its use in education demand a substantial 
investment by the university. The students and faculty of every discipline of the university community should 
benefit from planned improvements in IT infrastructure and a pedagogical-oriented instructional technology support 
system. The need for information technology (IT) infrastructure has become fundamental to the advancement of the 
academic enterprise and requires a holistic strategic planning to define the ideal instructional technology support 
model for university. 

This session described an institutional effort to recognize and resolve a significant problem in the 
implementation of new instructional technologies. At a research I public university, the Instructional Technology 
Council consisting of faculty members and technology services providers was charged to identify what are the 
appropriate developments necessary to provide faculty with a seamless teaching environment which includes 
technology based instruction both on the campus and at a distance. 

The issues were discussed thoroughly throughout the year among the council and faculty members. As a 
result, ten guidelines were identified and forwarded to the Dean’s Council for its consideration of formulating a plan 
to implement. This is a fist step of a holistic strategic planning to help faculty integrating technology in their 
teaching. This plan calls for a collaboration among campus offices and units to support faculty in their endeavors 
and recognize their needs for an easy transition of adapting technological changes in teaching and doing research. 

Ten guiding principles 

The ten guiding principles for integrating technologies in teaching are as follows: 

1. Definitions of scope, students, and technologies for the implementation 
To prevent repeated efforts or revamping the same technologies, the focus should be on one or two IT areas to 
develop at a time identify the target students who will benefit from the initiatives then devise a plan to achieve the 
objectives. 

2. Identify the supports to faculty 

• Offer both technological and pedagogical support 

• Develop local (departmental) support 

• Create users’ forum 

• Provide timely support (continuous availability) 

• Supports that meet the expectations 

3. Address issues of standardization 

Appropriate hardware needed by each faculty member and the students they teach may not be consistent with a 
common solution for the entire campus. A flexible approach to providing the technology to different disciplines 
must be combined with an understanding that all disciplines must be served by any mandate to increase IT. 
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4. Provide faculty equipment 

Some teaching pedagogy requires substantial investments in university owned and maintained equipment such 
as digital projectors and information servers, and in faculty training. Improvements in equipment may put 
economic pressures on some segments of our student body and may require special mechanisms to maintain 
equity. 

5. Establish intellectual property policy 

Lack of a universal policy on intellectual property policy caused unnecessary confusions and disputes between 
students and faculty among the use of classroom materials and sharing of students work on open public 
platform. A set of policy needs to be formulated and enforced encourage an open learning atmosphere and 
creative teaching and learning. 

6. Develop easy Web tools for course Web development 

An easy web tool that is intuitive for faculty to generate and publish course materials on the Web is helpful. A 
standard temple available to faculty will substantially ease the anxiety of creating individual tools and design. 

7. Realize not all subjects are suitable for distance learning 

University needs to realize there is no one size fits all solution when it comes to learning and teaching. It is 
imperative to be aware of the types of technologies available for on campus teaching as well as for off campus, 
and its implication for instructional design and development for different settings. 

8. Re-stmcture faculty workload, compensation 

Timely recognition of time and energy that faculty members have invested in teaching with technology is 
essential factor for faculty rewarding. The compensation mechanism needs to reflect the efforts of scholarship 
of teaching and development of innovative pedagogy for both traditional and non-traditional classes to maintain 
equity and morale. 

9. Strengthen teaching environment (less set up time) 

The current emphasis on maintaining support for focused IT programs need to be broadened to include an 
emphasis on increasing the number of IT-ready classrooms and providing a revenue stream that will provide 
appropriate computer equipment and training for all faculty and students. IT use on campus varies depending 
upon the equipment available in existing classrooms, the Internet connectivity, the training of the faculty and 
students, and the equipment available on faculty and student desktops. The slow increase in the number of well- 
equipped classrooms is a bottleneck to the general faculty's progress in developing pedagogy. 

10. Establish course administration policy and assessment standards. 

Administration policy and assessment procedures on technology-based courses should be in place and 
communicated on campus. Assessment practices need to address the nature of technology use and 
subsequently, the learning outcomes of various teaching/learning experiences derive from the new paradigm. 

Conclusion 

The definitive goals of creating a set of guiding principles for IT development are two folds; one is to 
formulate a systematic plan that is guided by the best practices and pragmatic preparation which are shaped 
profoundly by each campus’s culture and politics; also it leads to an ongoing dialogue between campus technology 
services providers and faculty, and among services providers to examine the approaches that each campus has taken. 
It is our hope that individual campus will apply these ten principles and to conduct similar needs assessment on their 
own campuses to identify the priorities for planning steps, establish the strategies and implementation guidelines. 
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What Should Instructional Technologists Know about Human Performance 

Technology? 
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James D. Klein 
Arizona State University 
Abstract 

Human performance technology (HPT) is having a significant impact on the field of instructional design and 
technology (IDT), and many IDT graduate programs now offer training in HPT to their students. Some IDT 
programs may be struggling with the extent to which they should incorporate the principles and techniques of HPT 
into their courses, however. To provide some determination of which specific skills and competencies in HPT 
graduates of IDT programs should have, an online survey was administered to 24 IDT faculty members and 45 
members of local chapters of the International Society for Performance Improvement and the American Society for 
Training and Development. Respondents rated the importance of HPT competencies for graduates of IDT programs. 
Results suggest strong support for the inclusion of HPT concepts in the curricula of IDT graduate programs. 
Implications of these results for IDT graduate programs, as well as for distinguishing HPT as afield of 
specialization separate from IDT, are discussed. 

What Should Instmctional Technologists 
Know about Human Performance Technology? 

Human performance technology (HPT) can be defined as . .a process of selection, analysis, design, 
development, implementation, and evaluation of programs to most cost-effectively influence human behavior and 
accomplishment” (International Society for Performance Improvement, 2002). It is an approach descended from 
systems theory, behavioral psychology, and instructional systems design (Rosenberg, Coscarelli, & Hutchison, 

1999). The advocates of HPT encourage those working in the field of instructional design and technology (IDT) to 
conduct broader analyses of performance and organizational systems. From the perspective of HPT, instruction or 
training is but one of many solutions available to improve human performance, and analysts should thus be prepared 
to consider, design, implement, and evaluate an increasingly varied array of non-instructional performance 
interventions. 

The HPT movement is having a significant impact on IDT (Reiser, 2001), and many IDT graduate programs 
have begun offering courses, special concentrations, or certificate programs in HPT. Indeed, a web-based review of 
the degree requirements and course offerings at 1 1 well-established IDT graduate programs (including Arizona State 
University, Boise State University, Florida State University, Indiana University, Pennsylvania State University, San 
Diego State University, Syracuse University, University of Georgia, University of Northern Colorado, Utah State 
University, and Wayne State University) reveals that eight of them offer at least one course specifically on HPT, and 
three offer more than one HPT course. In some programs, even the core instructional design courses have assumed 
an HPT orientation (Dick & Wager, 1998). Although most of the HPT courses offered by IDT programs are electives 
(only one of the programs listed above required an HPT course for its doctoral students, and only one required such 
a course for its masters students), it is clear that IDT faculty consider it important to offer some training in HPT to 
their students. 

The increased emphasis on HPT concepts and principles in IDT graduate programs is not surprising, 
considering the strong relationship and obvious similarities between the two fields. Given their traditional focus on 
training solutions, however, some IDT programs may be struggling with the extent to which they should expand their 
curricula to focus on HPT (Dick & Wager, 1998). Further, while the desired knowledge and skill sets of HPT 
practitioners has received some attention (e.g., Stolovitch, Keeps, & Rodrigue, 1999), as have the IDT competencies 
for instructional designers and trainers (e.g.. International Board of Standards for Training, Performance, & 
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Instruction, 2002), it is less clear which specific skills in HPT graduates of IDT programs should have. Concerns 
about the level of HPT knowledge needed by instructional designers and technologists, the degree to which a 
program can provide adequate training in both IDT and HPT, and the wisdom of broadening the field’s scope to 
include issues addressed by several other disciplines (such as human resources development, business management, 
and industrial/organizational psychology) are likely shared by many IDT faculty members. These are difficult and 
important issues, and how they are addressed will help shape the future of both IDT and HPT. 

The present study was conducted to provide a better understanding of what instructional designers and 
technologists should know about HPT. Such knowledge could provide guidance to IDT graduate programs seeking 
to prepare their graduates for the modern workplace and illuminate issues relevant to how professionals in both HPT 
and IDT are trained. To achieve these goals, a survey was developed to assess the attitudes and beliefs of 
professionals in the two fields about the HPT competencies graduates of IDT programs should have. The survey was 
administered to faculty members at IDT graduate programs throughout the United States, as well as members of local 
chapters of HPT and IDT professional societies. 

Method 



Participants 

IDT faculty members. One hundred one faculty members from the well-established IDT graduate programs 
listed earlier were invited via email to complete the web-based survey. Twenty-four faculty from nine different 
universities responded to the request, indicating a 24% response rate. Thirteen of these respondents were male 
(54%), 10 were female (42%), and one did not indicate his or her gender. Most (83%) of the faculty members indicated 
that they had more than 10 years of experience in IDT, and the vast majority (92%) rated their knowledge of IDT as 
“advanced.” Only 10 (42%) of the faculty respondents had more than 10 years of experience in HPT, and another 10 
(42%) reported having 5 or fewer years of experience. Most rated their knowledge of HPT as either intermediate (50%) 
or advanced (46%). 

Members ofISPI and ASTD. Members of the central Arizona chapters of the International Society for 
Performance Improvement (ISPI) and the American Society for Training and Development (ASTD) were also asked 
via email to complete the survey. Forty-five members of these organizations responded to the request. The overall 
return rate for this group cannot be calculated because contact was initiated via a listerv email message, and the total 
number of subscribers to the listserv was not available. It was estimated by a representative from the ISPI chapter, 
however, that several hundred members subscribe to the listserv. Of the 45 respondents, 29 were female (64%) and 
16 were male (36%). Most rated their knowledge of IDT to be intermediate (3 1 %) or advanced (5 1 %), and their 
knowledge of HPT to also be intermediate (38%) or advanced (44%). Members varied widely in their professional 
experience. With regard to years of experience in IDT, 11% reported having no experience, 22% reported 5 or fewer 
years, 18% reported 6-10 years, 22% reported I I-I5 years, 16% reported 16-20 years, and 11% reported 20 or more 
years. For years of experience in HPT, 11% reported having no experience, 31% reported 5 or fewer years, 20% 
reported 6-10 years, 18% reported II-I5 years, 11% reported 16-20 years, and 9% reported 20 or more years. 

Survey Instrument 

The HPT Competencies for Instructional Technologists Survey was a web-based instrument consisting of 
44 likert-type items and one open-ended question. Respondents received the following directions at the beginning 
of the survey: “Please rate how important you believe it is for graduates of instructional systems/design/technology 
graduate programs to have each of the competencies listed below.” Each Likert -type item consisted of a skill or 
competency for which respondents were asked to choose a rating from not important (scored as a 0) to very 
important (scored as a 4). The open-ended question asked respondents to provide any additional skills or 
competencies in HPT (not addressed on the survey) that they believed instructional technology graduates should 
have. 

The competencies listed on the survey were based primarily on a document analysis of the major topics and 
themes in the Handbook of Human Performance Technology (Stolovitch & Keeps, 1999) and the content and 
competencies of the syllabi for HPT courses offered by several IDT programs. Rather than list the dozens and dozens 
of possible performance interventions on the survey, some of the general intervention categories presented by 
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Hutchison and Stein (1998) were used in constructing the competencies. For each intervention category, two items 
were written: one addressing knowledge of and familiarity with interventions in that category, and the other 
addressing skill in developing and implementing the interventions in that category. This distinction between 
knowledge and skill was made because interviews with faculty members revealed that some believe it is imp ortant to 
be aware of a wide range of performance interventions, but that skill in actually developing and implementing all of 
the interventions is not necessary. 

Results 

Respondents to the HPT Competencies for Instructional Technologists Survey deemed most of the 
competencies to be at least “somewhat important’’ for graduates of instructional technology programs, and the two 
respondent groups (ISPI/ASTD members and IDT faculty) rated most of the items similarly. Table 1 displays the 
means scores for each of the survey items, listed in order of highest to lowest overall mean, with scores calculated 
using the Likert -type scale: 4 for a rating of very important, 3 for a rating of important, 1 for a rating of somewhat 
important, and 1 for a rating of not important. 

Table 1 

HPT Competencies for Instructional Technologists Survey Mean Ratings 



Rank ISPI/ASTD IDT Overall 

Number Item Statement Members Faculty Mean 

(n=45) (n=24) (n=69) 



1 Distinguish between performance problems requiring 



instructional solutions and those requiring non- 3.87 

instructional solutions 

2 Conduct a performance analysis for a specific 

situation to identify how and where performance 3.82 

needs to change (the performance gap) 

3 Evaluate a performance improvement intervention to 
determine whether or not it solved the performance 3.86 
problem 

4 Conduct a cause analysis for a specific situation to 
identify factors that contribute to the performance 3.78 

gap 

5 Select a range of possible performance interventions 

that would best meet the need(s) revealed by the 3.76 

performance and cause analyses 

6 Assess the value of a performance improvement 

solution (in terms of return on investment, attitudes 3.74 

of workers involved, client feedback, etc.) 

7 

Define and describe human performance technology 3.58 



3.96 3.90 

3.79 3.81 

3.63 3.78 

3.64 3.74 

3.63 3.72 

3.54 3.67 

3.75 3.64 



8 Identify and implement procedures and/or systems to 

support and maintain performance improvement 3.60 3.38 3.52 

interventions 
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9 



10 



11 



12 



13 

14 



15 



16 



17 

18 



19 



20 



21 

22 



Describe and be familiar with (NOT imp lement) a 
variety of performance interventions in the area of 
measurement and evaluation 

Develop and implement a variety of performance 
interventions in the area of measurement and 
evaluation 

Describe the general model of human performance 
technology (the systematic combination of 
performance analysis, cause analysis, and 
interventions selection) 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
instructional technology 

Develop and implement a variety of performance 
interventions in the area of instructional technology 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
feedback 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
job and workflow 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
organizational design and development 

Develop and implement a variety of performance 
interventions in the area of feedback 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
communication 

Develop and implement a variety of performance 
interventions in the area of organizational design and 
development 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
quality improvement 

a Develop and implement a variety of performance 
interventions in the area of job and workflow 

Describe and be familiar with (NOT implement) a 
variety of performance interventions in the area of 
reward/recognition 



3.59 

3.53 

3.49 

3.40 

3.40 

3.33 

3.40 

3.25 
3.33 

3.23 

3.26 

3.24 
3.35 
3.17 



3.35 

3.42 

3.42 

3.46 

3.46 

3.38 

2.92 

3.17 

3.04 

3.08 

3.00 

2.96 

2.67 

2.78 



3.51 

3.49 

3.46 

3.42 

3.42 

3.34 

3.22 

3.22 

3.22 

3.18 

3.16 

3.14 

3.10 

3.03 
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23 



24 

25 

26 



Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 3.05 

information 

Develop and implement a variety of performance ^ 

interventions in the area of quality improvement 

Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 3.09 

documentation and standards 

Develop and implement a variety of performance ^ 

interventions in the area of communication 



Develop and implement a variety of performance 
interventions in the area of information 



28 



29 

30 



31 



32 



33 

34 

35 

36 



37 



Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 3.05 

human development 

Describe the history and conceptual underpinnings ^ ^2 
of human performance technology 

Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 2.90 

management science 

Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 2.91 

resource systems 

Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 3.02 

selection 

Develop and implement a variety of performance 
interventions in the area of documentation and 3.02 

standards 

Develop and implement a variety of performance ^ 

interventions in the area of reward/recognition 



Identify the similarities and differences among a 2 g 2 

variety of specific performance technology models 

Describe a variety of specific performance 

technology models (e.g., those of Mager, Harless, 2.70 

Rummler & Brache, Tosti & Jackson, Langdon, etc.) 

Develop and implement a variety of performance 2 gg 

interventions in the area of human development 



3.00 

2.71 

2.75 

2.61 

2.87 

2.67 

3.13 

2.59 

2.59 

2.42 

2.42 

2.33 

2.54 

2.71 

2.42 



3.03 

2.98 

2.97 

2.94 

2.94 

2.91 

2.80 

2.80 

2.80 

2.80 

2.80 

2.75 

2.72 

2.71 

2.71 
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38 



39 

40 



41 



Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 2.73 

career development 

Develop and implement a variety of performance 2 

interventions in the area of management science 

Describe and be familiar with (NOT implement) a 

variety of performance interventions in the area of 2.60 

ergonomics 

Develop and implement a variety of performance 2 g 2 

interventions in the area of selection 



2.30 2.58 

2.43 2.58 

2.52 2.57 

2.13 2.56 



42 

43 

44 



Develop and implement a variety of performance 
interventions in the area of resource systems 

a Develop and implement a variety of performance 
interventions in the area of career development 



Develop and implement a variety of performance 
interventions in the area of ergonomics 



2.22 2.55 

1.88 2.35 

1.96 2.18 



Note. 4 = very important, 3 = important, 2 = somewhat important, 1 = not important 
‘‘Ratings of ISPI/ASTD members are significantly higher than the ratings of IDT faculty. 

Independent t-tests were conducted on each of the 44 survey items to test for significant differences 
between the two respondent groups. With an alpha level of .05 and using the Bonferroni correction procedure to 
compensate for the large number of comparisons, significant differences between the two groups were found for only 
2 of the 44 items. The competencies for which significant differences were found include “develop and implement of a 
variety of performance interventions in the area of career development’’ and “develop and implement a variety of 
performance interventions in the area of job and workflow.’’ The members of ISPI and ASTD rated both of these items 
as more important (M = 2.6 andM = 3.3, respectively) than did the IDT faculty (M = 1.9 andM = 2.7, respectively). 

Due to the largely similar responses of the two groups, only the means of the combined scores of the two groups will 
be examined hereafter. 

All of the items received an overall mean score of at least 2.0, corresponding to a ranking of somewhat 
important. In addition, 23 of the 44 competencies received an overall mean score of 3.0 or higher, indicating an 
average ranking between important and very important. The competency rated the most important overall was 
“distinguish between performance problems requiring instructional solutions and those requiring non-instructional 
solutions’’ (overall M = 3.90), followed by “conduct a performance analysis for a specific situation to identify how 
and where performance needs to change (the performance gap)” (overall M = 3.81). The remaining highly -rated 
competencies cover all phases of the performance improvement process, including performance analysis, cause 
analysis, intervention selection, intervention implementation, and performance evaluation. The two lowest-rated 
items were “develop and implement a variety of performance interventions in the area of ergonomics” (overall M = 
2.18) and “develop and implement a variety of performance interventions in the area of career development” (overall 
M = 2.35). Interestingly, the competencies associated with knowledge of different HPT models (“describe a variety of 
specific performance technology models” and “identify the similarities and differences among a variety of specific 
performance technology models”) did not receive mean scores above 3.0 (although the means for both did rank 
somewhere between somewhat important and important). 

Only 28 of the 69 total respondents provided an answer to the survey item asking if there were any 
additional skills or competencies in HPT they believed instructional technologists should have, and there was a great 
deal of variability in their answers. A number of them listed skills which were relatively vague and did not seem to be 
specific to HPT, such as “interpersonal skills,” “organizational and diplomatic skills,” and “cultural sensitivity.” 
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General topic areas that were mentioned by several of the respondents included communication and writing, project 
management, the systems approach, computer technology, and needs assessment. 



Discussion 

Overall, this sample of professionals and academics in the fields of IDT and HPT felt that graduates of IDT 
programs should have a broad knowledge of HPT and the performance improvement process. Every single 
competency listed on the survey received an overall mean rating of at least somewhat important, and over half of the 
items were rated as important or higher. These results indicate strong support for the incorporation of HPT concepts 
and techniques into the curricula of IDT graduate programs. 

Many of the highly -rated competencies reflect skills and knowledge that are likely already addressed in most 
IDT programs. The two highest-rated competencies, for example, are both closely related to traditional training needs 
assessment, a staple of most IDT curricula. Some of the highly -rated competencies, however, probably do not receive 
extensive coverage in most IDT programs. Competencies such as “select a range of possible performance 
interventions that would best meet the need(s) revealed by the performance and cause analyses’’ and “identify and 
implement procedures and/or systems to support and maintain performance improvement interventions’’ may not be 
emphasized in many IDT programs, particularly in those that do not offer HPT courses. In addition, many of the 
competencies related to specific performance interventions are probably not covered outside of courses focusing 
specifically on HPT. Given that most IDT programs do not require HPT courses of their students, there may be 
numerous discrepancies between the curricula being offered by these programs and the competencies the 
respondents to this survey consider important. 

Not surprisingly, the lowest-rated competencies were those involving the development and implementation 
of specific performance interventions, such as ergonomics or career development. Most professionals recognize that 
“practitioners are not expected to be experts in all categories and subcategories of interventions’’ (Van Tiem, 

Moseley, & Dessinger, 2000, p. 64), and that the expertise of other members of an organization will often be called 
upon to implement interventions. Nevertheless, the relatively high ratings for all of the survey items indicates that 
those working in IDT and HPT consider it important for instructional designers and technologists to be at least 
familiar with a variety of performance improvement interventions. 

Despite its limited sample size, the present study clearly reveals some support in both academia and 
industry for training IDT students in the principles and methods of HPT. These findings also contribute to the 
confusion that frequently surfaces in distinguishing HPT from other fields, such as IDT (Stolovoitch, Keeps, & 
Rodrigue, 1999). If IDT programs continue incorporating more and more of the HPT perspective into their curricula, 
and if IDT programs are one of the primary sources for graduate-level training in HPT, is it truly useful to view HPT 
as a separate field of specialization from IDT? Is it more appropriate to speak of IDT as a subset of HPT, as some 
suggest (Hutchison, 1990)? The present results also lend weight to the concerns of Dick and Wager (1998), who 
wonder whether IDT programs can handle the additional responsibilities of teaching HPT or if such programs will 
need to be split into more specialized areas. And if IDT programs do not expand their curricula to include more 
extensive coverage of HPT, where will individuals receive professional training in HPT? Future research extended to a 
larger sample of the professional population, as well as a more detailed analysis of the degree to which HPT is 
currently covered in the curricula of IDT programs, may provide answers to some of these questions. Only time will 
tell, however, if HPT is best viewed as a new field that revolted against the limited scope and focus of IDT, or simply 
a new perspective that evolved within IDT. 
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The Virtual Factory Teaching System (VFTS) project is a web-based, multi-media collaborative learning network. 
The system allows students, working alone or in teams, to build factories, forecast demand for products, plan 
production, establish release rules for new work into the factory, and set scheduling rules for workstations. This 
paper provides a system description and a discussion of the evaluation component involving three different 
campuses. Evaluation results cover demographic descriptions, self-assessment results, performance results, 
attitudinal responses, and usability. In addition, instructor observations and course project output is also 
examined as components of the VFTS evaluation effort. It is believed through the analysis that the VFTS is an 
excellent instructional method to teach students the integration of the different modules in operations planning. 

Background 

Traditional pedagogical tools for transferring hands-on learning to students are ill-equipped to handle the 
complexity that surrounds the modern factory. Manufacturing education tools have traditionally required physical 
laboratories (e.g., machine shops). However, factory experimentation through full-scale on-campus laboratories is an 
infeasible alternative due to the high expense associated with development and maintenance, and the inability to 
cover the entire manufacturing spectrum. Additionally they can serve only a single site, inhibiting the creation of 
teams whose members are not co-located, i.e., "virtual" teams which are increasingly becoming a requirement for 
engineers. 

A web-based, multi-media collaborative learning network, referred to as a Virtual Factory Teaching System 
(VFTS), was developed under two National Science Foundation (NSF) grants (Dessouky, et. al 1998; Dessouky, et. al. 
2001; Kazlauskas, et. al. 2000; Kazlauskas, 2001). The first grant was a planning and demonstration grant with a full 
implementation and evaluation the essence of the second grant. This tool is currently being used by engineering 
students from the University of Southern California, San Jose State University, the University of Virginia, and most 
recently North Carolina Agricultural and Technical State University. The overall aims of the project are: 

• To provide, disseminate, and evaluate a manufacturing education pedagogical tool that promotes 
student understanding of complex factory dynamics. 

• To improve student skills in communication, persuasion, negotiation, and management, as well as in the 
technical arenas of production planning, forecasting, simulation, scheduling, and integration. 

• To provide a forumin which engineering and business school students can participate in virtual teams 
that cut across universities. 

The research plan of the project is aimed at exploring the interface between virtual factories, engineering 
education, intelligent agents, and the Internet for new ways of teaching modern manufacturing problems, practices, 
theory, and techniques to engineering and business undergraduate students. In addition, it is aimed at examining its 
potential for use as an information vehicle on the topic of manufacturing for K-12 students. Various research 
questions are addressed, such as: how students perform when using new technology-enhanced modes of learning; 
what are the effects on attitudes; how intelligent agents might assume tutoring and participative roles; and how team 
performance, hampered by geographical separation, might be enhanced via advanced communication technologies. 
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The project timetable consisted of the following: a Baseline period to provide an understanding of the 
Industrial Engineering courses (with scheduling content) using traditional instructional methodologies, i.e. without 
the VETS; the use of the VETS to provide an understanding of the Industrial Engineering courses (with scheduling 
content) offered at the various institutions; then the use of an expanded version of the VETS at the same institutions 
with pedagogical agents and use of virtual teaming included. The following presents an overview of the features of 
the VETS; a description of the evaluation approaches; and three-year project results. 

System Description 

The architecture of the VETS was kept simple and modular. There are three layers in the design: AweSim 
Server, VETS Java Server and Clients. Clients, which are students in our case, use standard WWW browsers like 
Microsoft Internet Explorer, to connect to the VETS Java Server using its Web Page. Most of the communication 
between the clients and the server takes place using Java applets. The Java-Server functions as a mediator between 
the AweSim factory servers and the clients. The Awesim Server is responsible for factory knowledge and simulation. 
The layers interface using a message protocol set up to minimize bandwidth requirements. Other components of the 
VETS include the use of Ptolemy, a graphing package, and Drasys, an Operations Research (OR) package. 

The system allows students, working alone or in teams, to build factories, forecast demand for products, 
plan production, establish release rules for new work into the factory, and set scheduling rules for workstations. 

They can run simulations where an animated panel displays jobs progressing through their factory, with queue 
counts, finished goods counts, graphs, and reporting functions all available. Students access via the VETS, using 
computers in university comp uter labs, or in their dorms or homes, a virtual representation of the factory. The 
professor posts assignments related to this factory over the Internet, “unfreezing” parameters as necessary so 
students may experiment without redefining the entire factory. Students observe the effects of their decisions, and 
student teams assume factory roles to solve problems; if they reach an impasse, intelligent agents provide guidance. 
Selective information may be given based on student roles; for example, the production supervisor may have 
equipment information, the engineer new technology information, etc. Students sort out strategies and can discuss 
options via e-mail and electronic chat rooms. Since this course is a common one found in many universities, 
collaboration among universities is feasible. Eaculty members virtually “team teach” the course. Intelligent agents are 
incorporated into the VETS to monitor student progress and provide immediate feedback. 

The latest version of the VETS software includes support for user account management, adding security to 
allow students within a group to share data while preventing access to it from students outside the group, a more 
easy to use interface, instrumentation so that the software will gather data on student usage patterns, on-line help, 
on-line documentation and an on-line tutorial to help for students and faculty learn how to use the VETS, and an 
introductory homework exercise to help students learn how to integrate the VETS into the course. A project that 
uses the VETS was developed to complement the students' classroom learning and was integrated into the courses at 
the participating universities. The latest version also includes a pedagogical agent that monitors students' use of the 
VETS and provide guidance. This required integrating the pedagogical agent software (ALI) into the VETS, adding 
the ability for ALI to maintain a persistent model of each student's knowledge across sessions, extending ALI to 
include more sophisticated explanation capabilities, adding VFTS-specific knowledge to allow ALI to understand the 
instructional objectives of the VETS and provide appropriate support to students, and instrumenting ALI to maintain 
a log of interactions with students to aid our evaluation. 

The current version VETS is available at http://vfts.isi.edu . 

Project Tasks 

The VETS project included a large set of tasks involving design, development, usage, evaluation, and 
dissemination. The various tasks associated with the VETS project are as follows: 

■ Define instructional objectives and complete evaluation design. 

■ Develop and/or acquire evaluation instruments. 

■ Solicit feedback on the instructional objectives, evaluation design, and VETS use from each university. 

■ Gather evaluation data for control group (engineering classes without the VETS). 

■ Analyze evaluation results for control groups. 

■ Complete instrumentation of the software to support evaluation 
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■ Teach engineering classes with the VFTS and gather evaluation data. 

■ Analyze evaluation results for experimental group and compare to control group. 

■ Use the evaluation results to revise and refine the VFTS and its use in the engineering curricula, its 
instrumentation, and the pedagogical agents. 

■ Teach engineering classes with the VFTS, including the use of virtual teams that span multiple 
universities, and gather evaluation data for this final experimental group. 

■ Analyze summative evaluation results. 

■ Make final revisions to the VFTS software based on evaluation results. 

Evaluation 

In the preceding list of project tasks, it should be noted that considerable emphasis is placed on the design 
and implementation of an extensive, multi-university evaluation, which is a central component of the VFTS project. 
The initial evaluation efforts were used a) to check to make sure each project step was implemented according to plan 
and that milestones are being met; and b) to consider project modification. This evaluation, formative in nature, 
assisted in modifying the design and development of the evaluation, as well as of the overall design of the web- 
based VFTS. For example, a Computer Competencies Multiple Domains instrument that was used initially to 
determine the level of entering competency in such areas as spreadsheets and statistical packages was dropped. It 
was determine through initial surveys that the learners were homogeneous in terms of a high level of computer 
competencies. Through the evaluation process, it was decided to gather more data on such issues on the ease of use 
and operation of the web-based VFTS, and to modify the actual design by changing the opening screen, and by 
including more tutorial help. 

An emphasis of this project is to evaluate learner outcomes, attitudes, and learning. This effort includes the 
use of various instruments and examination of student work. 

The following are the instruments that are presently being used in evaluation: 

• Student Pre -course and Post-course Survey instruments which are used to gather data on student 
demographics and student self-rated assessment of entering/post-course knowledge of operations 
scheduling, such as the skills, knowledge, and abilities in the use of gantt charts and regression and 
time series models. 

• Computer Competencies Multiple Domains instrument which is used to determine the level of entering 
competency in such areas as spreadsheets and statistical packages. 

• Operations Scheduling Pre-test/Post-test instruments which are used measure course content before 
and after instruction and use of the VFTS. 

• Affective-Level/ Attitudes Instrument (VFTS Participant Opinion Survey) Pre-test/Post-test instrument 
which are used to measure reactions to the use of the VFTS, such as reactions to use of simulations, 
working collaboratively, and relevance to future career. 

• Usability Evaluation instrument which is used to gather data on such issues on the ease of use and 
operation of the web-based VFTS. 

The VFTS Studies 

Various studies were conducted in the VFTS Project. The first of these was the Baseline evaluation 
conducted at the various universities, to provide an understanding of the Industrial Engineering courses (with 
scheduling content) using traditional instructional methodologies. The second of these was an investigation of the 
use of the VETS at the same universities, to provide an understanding of the Industrial Engineering courses (with 
scheduling content). A second usage of the VETS occurred, an enhanced version with teaming and pedagogical 
agency. The following represents the findings from these investigations at three universities to-date since data are 
still being collected at one of the institutions. 

The following represents a review of the comparisons among the various institutions involved in the 
Baseline, VFTS, and enhance version of the VFTS with agency and teaming. 

• The number of subjects in the courses at two institutions varied from 15 to 44, whereas a third institution 

had a larger number of subjects (high 60’s) for the Baseline yet a low number (6) for the first round of the 
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VFTS. For the most part, there was minimal leakage of students from the class or in survey response 
numbers. 

• The courses were all taught by experienced, tenured faculty but the course was a new offering for each of 
the faculty at two institutions. 

• Most students were in the 21-22 age category. Flowever, the age distributions at two institutions were more 
typical of a ‘traditional’ undergraduate student, whereas the students at one institution were older. 

• Students were mostly seniors. 

• The majority of students at all three campuses were male. A Chi-Square test on the gender variable was 
performed to determine if there is a significant difference between the groups. There is no significant 
difference on this variable. 

• The primary language at all three institutions was English but 19 other languages represented approximately 
40%to 60% of the students at two institutions. The students at a third institution were predominately 
English-speaking. The assumption is made that there are more international students at institutions #1 and 
# 2 . 

• Students at all three institutions had a moderate level of interest in the course. They rated themselves low in 
knowledge of course content. The results of a corresponding post-course self-assessment of content 
knowledge indicated gains in most areas but not quite to the significance level. Note that there was no post- 
self assessment survey at institution #3. Results of a self-assessment indicated a high-level of student 
computer competency at all three institutions. The results of an initial course opinion survey indicated fairly 
high ratings on interest, importance, confidence, usefulness and similar scales on such topics as simulation, 
and collaboration. There was no significant change on post-course opinion survey. Note that there was no 
post-self opinion survey at institution #3. 

• With a few exceptions, one can support the general statement that students know little, if anything, about 
the content of the course but think that it will help them in a more general way. Students at two institutions 
indicated that they did see the course assisting in developing fundamentals, e.g. improving skills in 
modeling-building and analysis and assisting in working better with fellow students, whereas one institution 
did not have this self-assessment. 

• The Computer Competencies Multiple Domains instmment indicates a very high level of skill with computer 
technology. The only questions that did not reflect this generalization deal with some programming skills 
and statistical analysis. Due to the high self-assessment and the homogeneity of the students, the use of 
this instrument was dropped. 

• The gains from a course content pretest to posttest were significant at all three institutions. The mean 
pretest to posttest scores at institution #1 for VFTS Year 1 were 12.95 to 41.52, and 16.9 to 45.9 for VFTS 
Year 2; the scores at institution #2 for VFTS Year 1 were 8.00 to 20.86, and 8.8 to 25.5 for VFTS Year 2; and 
the scores at institution #3 were 10.4 to 87.6. Separate final examinations were administered at two 
institutions. Only at institution #3 was the posttest a component of the final grade. The mean at institution 
#1 was 85.55 for Year 1 and 85.4 for Year 2 out of a score of 1 10; the mean at institution #3 was 83.5 for Year 
1 and 81.1 for Year 2 out of 100. It should be noted that the institution #1 final covered more than machine 
scheduling content. The grade distribution at this institution indicated that 50% of the students received a 
grade of B or better; slightly less that 50% received B or better at institution #2. No grades were provided for 
institution #3. 
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• Results of the usability assessment indicated that the forecasting, planning, and scheduling modules were 
fairly well-received as was the factory simulation. The online documentation and tutorials were not used or 
found of limited value. Initially, the software was not seen as particularly “user friendly’’ with problems 
associated with crashing systems, and slow response. But, it was determined that the initial criticism from 
the students in regards to using the VFTS were not tool-related but Internet-related, e.g. the server was not 
available or too slow. Such issues have been addressed. Students were somewhat neutral to positive 
towards use of agents, and negative towards use of virtual teams. 

• Comments from instructors noted that the VFTS was instrumental in facilitating the students' understanding 
of the integration between the models for forecasting, production planning, material planning, inventory 
planning, and scheduling. This was illustrated by the fact that students clearly spent a good part of their 
time testing many different scenarios, and students were able to see the impact of changing a parameter on 
the output. It is suggested that a more complex system can be modeled than otherwise. Instructors also 
noted that due to the VFTS the class was able to work on a more comprehensive project (when compared to 
class not using the VFTS) that illustrates an integrated dynamic factory. 

The result of various analyses on the data from the 1) Baseline to VFTSl Pre -Course Instmment Comparison 
for the three universities; 2) Baseline to VFTS2 Pre -Course Instrument Comparison for institutions #1 and #2; 3) 
VFTSl to VFTS2 Pre-Course Instrument Comparison for institutions #1 and #2; 4) Post Course VFTS Instrument 
Comparison for institutions #1 and #2; 5) Pre-Test and Post-Test Operations Scheduling Comparisons for all three 
institutions; and 6) Final Examination Comparisons for institutions #1 and #2 indicated a number of patterns. A 
pattern of significant differences exists between the results on the Pretest for institutions #1 and #2 Baseline and the 
pretests for both institutions for the VFTSl as well as VFTS2. The institution #3 Baseline posttest and VFTSl was 
significant. We can draw some general findings from these analyses. First, there is no significant difference in the 
final examination scores at institutions #1 and #2 between the baseline and the VFTS scores. This result might be 
somewhat expected since the final examination was based on standard planning and scheduling questions and 
concepts that might not require a dynamic simulation model to illustrate the material. Since the final examination is a 
large portion of the final grade, the study team did not want to provide any examination questions that would be a 
disadvantage in terms of grade to the baseline group. Second, the baseline group scored higher on the pre-test at 
both institutions #1 and #2 than their counterparts in the VFTS group, particularly at institution #1. However, since 
both groups had the same final examination scores, this could imply that the less strong content-knowledge groups, 
using the VFTS, performed equivalent to the stronger content-knowledge groups. The VFTS might have some overall 
effect on final examination performance.. Third, within all groups, students scored higher on the post-test as 
compared to the pre-test, which is expected since the post-test was administered after the scheduling material was 
covered. Fourth, the post-test scores are significantly higher at institution #3 than at the other universities. We 
remark that at only this institution was the post-test a component of the student's final grade. In fact, it was treated 
as the final examination so in this case the students took it much more seriously, where they most likely studied for 
the examination. As a result, the only baseline group in which the post-test was administered was at institution #3. 

A comparison of the post-test scores between the baseline and VFTS groups at the institution #3 shows a significant 
gain for the VFTS group, although for a much smaller class size. 

Other Results 

The NSF funding for this project has supported five graduate students at the University of Southern 
California. These students have gained valuable experience in Web-based software development, user interface 
principles, pedagogical agent technology, educational materials development, and educational evaluation. A VFTS 
competition workshop among various institutions in the southwest was also held and students commented 
positively on the learning experience. But perhaps the most valuable training for all the students is the close 
multidisciplinary collaboration between computer scientists, engineering educators, and educational researchers and 
educators. These students will have a much broader perspective on research than typical graduate students that 
work closely with others only in their own field. 
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Conclusion 



The VFTS was instrumental in facilitating students' understanding of the integration between the models for 
forecasting, production planning, material planning, inventory planning, and scheduling. 

It was demonstrated through an analysis of project reports that students had a better understanding of the 
integration between the different modules. This was illustrated by the fact that students clearly spent a good part of 
their time, with the VFTS, in testing many different scenarios, and students were able to see the impact of changing a 
parameter on the output. It is suggested that another major benefit of the VFTS is that a more complex system can be 
modeled than otherwise. Thus, it was seen that students had a more realistic project than their counterparts in the 
Baseline groups. To summarize, it is believed through the analysis that the VFTS is an excellent instructional method 
to teach students the integration of the different modules in operations planning. 
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Customizing instruction to meet individual needs is one of the foundational cornerstones of today’s learner- 
centered paradigms. Adult learners have a wide range of differences in their backgrounds, interests, abilities, and 
learning styles; instruction, therefore, needs to be designed in such a way as to meet the highly diverse needs of 
adult education settings. The World Wide Web presents enormous potential for providing a technological 
environment for the optimal delivery of personalized instruction for individual learners. Some argue, however, that 
most Web-based instruction fails to effectively customize instmctions for individual learners. Therefore, we are in 
need of an ongoing refinement and creativity in our generation and treatment of theories of instruction geared 
towards the generation of personalized learning environments. 

Here, the attempt has been made to develop an instructional theory for personalized learning for online adult 
learners, with a special focus on the question of solving ill-structured problems. Theory, on a general level, is 
discussed insofar as it has emerged from the goals, preconditions, and values of instruction, with an eye to the 
methods of instruction that are optimal for achieving the goals. These theoretical frameworks serve as a basis for 
fostering problem solving skills in ill-structured domains in online learning environments, as seen in the literature on 
teaching problem-solving skills concerned with constructive and interactive learning environments and the use 
animated pedagogical agents. Goal-setting activities, engaging in the task, performing the task, and reviewing and 
reflecting upon the output of the task are referred to as the ‘methods’ of this instructional-design theory. In addition, 
some caveats in implementing this instructional-design theory and ways to evaluate and improve it are described in 
the conclusion. 



Introduction 

Customizing instruction to address the individual difference in learning needs is one of the key markers of 
the learning-focused paradigms of today’s education and training. Reigeluth (1999) has convincingly argued that 
social changes in the information age call for fundamental changes in the current paradigms of education and 
training. Thus, he claims that one of the key markers of the industrial age was standardization for mass production 
and it is shifting toward customization in the information age (Reigeluth, 1997). Therefore, “customization” of 
instruction is a key characteristic of the new paradigms of instruction in the information age, wherein the goal of 
instmction is to help individual learners reach their full potential. 

In particular, adult learners have a great volume and different quality of experience throughout their lives 
with a wide range of differences in their backgrounds, interests, abilities, and learning styles (Knowles, 1984). 
Therefore, an emphasis needs to be placed on adult learners to provide them with personalized instruction that 
addresses individual differences in their learning needs (Bond, 1992; Dunn, 1984; Martinez, 2000). Adult learners also 
need flexible learning environments to accommodate their learning with their busy schedules and multiple 
commitments at work, home, and the community. Personalized instruction allows more flexibility in the pace of 
learning than do group-based approaches such as lectures or small-group teaching (Bond, 1992). 

Web-based instmction seems fit the needs of the flexible learning environment for adult learners (Digilio, 
1998), and due to this, more adults are turning to online learning for their learning needs to keep up with rapidly 
changing knowledge and skills in the information age (Jarvis, 1999, 2001). Also, research suggests that Web-based 
instruction permits learners to control their own learning (Bonk & Cummings, 1998; Yong, 1998), adapt to different 
individual learning styles (Muir, 2001), and make pedagogical choices (Arnone, 2002). Therefore, the Web presents 
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enormous potential for the optimal delivery of personalized instruction for individual learners (Digilio, 1998; Martinez, 

2000 ). 

Despite the potential of the Web for the optimal delivery of personalized instruction, the authors would like 
to claim that most Web-based instruction, or online courses, offered nowadays are not effectively designed to 
provide instruction customized for individual learners. They are often designed from the personalized instruction 
approach in the sense that they provide self-paced instruction in which the learner can skip some of the units that 
s/he has already mastered prior to instruction. However, current online courses do not provide learners the locus of 
control - i.e.. All the learners are given instruction with the same learning goals and the same methods to achieve 
them. Thus, this instructional approach does not adequately address the issue of personalizing instruction for the 
individual learner. In addition, Arnone (2002) and Martinez (2001) argue that current Web-based instructions are not 
addressing the issue of differences in individual learning styles and preferences (Arnone, 2002; Martinez, 2001). 

In providing personalized instruction for adult learners, two factors need to be taken into account. First, 
theories of adult learning inform us that adult learners make sense of what they learn based on their experiences 
(Knowles, 1996). Also important is the relevance of learning to the learner which influences learner motivation 
(Keller, 1983). Therefore, the goals of learning for adult learners need to be closely related to their needs in real-world 
situations in order for them to actively engage in meaningful learning. Second, as mentioned earlier, differences in 
learning styles and preferences of individual learners need to be considered in designing personalized instruction. 
Thus, learners need to have control of how to learn in order to provide instruction that meets individual learning 
needs. 

In this paper, an instructional-design theory for personalized learning in online learning environments for 
adult learners is proposed. According to the instructional-design theories introduced in Reigeluth (1999), the initial 
step in the development of an instructional theory is to identify its goals/purposes, preconditions and values, which 
are discussed in the following. 

Goals / Purposes 

The primary purpose of this theory is to identify the conditions and methods that are necessary for 
personalized learning in online settings for adult learners, and how it can be best facilitated. This instructional theory 
is centered on fostering higher-order thinking skills, in particular ill-structured problems are of major focus of this 
theory. Today’s workforce often deals with ill-structured problems that are not clearly defined and new solutions 
need to be created. As Fred Nickols (Gordon & Zemke, 2000) suggests, in the new economy more and more jobs are 
shifting from “preconfigured work,’’ where the job is clearly laid out to “configured work,’’ where it has to be made up 
as it goes along. Therefore, instructional theories to teach problem solving skills on ill-defined problems are 
warranted in order to facilitate the development of competent knowledge workers. 

Preconditions 

Preconditions of an instructional theory refer to the instructional conditions that need to be met in order for 
the theory to be applied in real instructional situations. Those conditions include (1) the nature of the learning 
domain to be learned, (2) the nature of the learner, (3) the learning environment, and (4) any constraints for 
instructional development (Reigeluth, 1999). It is important to identify preconditions of an instructional theory since 
an instructional method which works well under one condition may not work well under another, which Reigeluth 
(1997) refers to as “conditionality.’’ Therefore, identifying conditions under which an instructional theory can work 
well is important to make the theory more likely to be useful (Reigeluth, 1997). 

It is suggested that the following preconditions need to be taken account to make a decision whether the 
instructional theory suggested here is suited for the instruction that you plan to design or teach; 

• Its methods are for individualized instruction especially in online learning environments - not intended for 
group instruction in a traditional classroom setting. 

• This theory is targeted for teaching cognitive domains, in particular problem-solving skills for relatively ill- 
structured problems. Therefore, it is not suited for other kinds of learning domains - i.e., psychomotor or 
affective domains. 

• An animated pedagogical agent is available for social interactions with the learner. 

• A learning management system (LMS) is available to help learners to manage their learning process. 



253 




• Learning objects are available for individualized and just-in-time access to knowledge to be acquired to go 
through the learning process. 

Values Reflected in This Theory 

According to the systems design theory by Banathy (1991), the design of an educational system is 
grounded on the values that are shared by the stakeholders and end users. Reigeluth (1997) also argues that values 
play at least two important roles in designing an instruction - i.e., values make an influence on the learning goals that 
are selected and they also influence the means for attaining a given goal, therefore laying a ground for the criteria 
against alternative means to accomplish a goal. 

The following values are identified as the criteria for the goals and instructional approaches that are adopted 
in this instructional theory. 

1. Learners should be able to learn at their own pace, at any time and at any place. 

2. Learners should be active, self-directing in pursuing their learning goals. 

3. Learners are motivated to learn when they experience in their life situation “a need to know or be able to do 
in order to perform more effectively and satisfyingly” (Knowles, 1996. p.256). 

4. Learners should be taught in a way that they could link new knowledge to their prior 
knowledge/experiences. 

5. The learners should be taught in a way that they could transfer knowledge into their real-life situations. 

6. Learners need to explore cross-disciplinary subjects in order to acquire knowledge and skills for solving 
real-world problems. 

7. Learners have options to choose the methods of their instruction - “students as designer” (Reigeluth & 
Nelson, 1997). 

8. Instructors need to act as mentor or guide to the learners helping them become self-directed learners. 

Theoretical Framework 

Before we discuss the methods that are employed to develop a new instructional theory for teaching solving 
Hi-defined problems, it is important to address the theoretical framework upon which the methods are developed. 
Those theoretical frameworks are discussed here with a focus on the theories of instructional strategies for guiding 
and coaching individual learners to promote their problem solving skills . 

Definition and Characteristics of Ill-Stmctured Problems 

Jonassen (1997) defines well-structured problems as “constrained problems with convergent solutions that 
engage the application of a limited number of rules and principles within well-defined parameters” (p.65). In contrast. 
Hi-structured problems “possess multiple solutions, solution paths, fewer parameters which are less manipulable, and 
contain uncertainty about which concepts, rules, and principles are necessary for the solution or how they are 
organized and which solution is best” (Jonassen, 1997, p.65). Ill-structured problems are typically situated and 
emerged from a specific context that is encountered from everyday practice (Jonassen, 1997). In situated problems, 
the problem situations are not well specified, the problems are not clearly described or well defined and the 
information needed to solve them is not contained in the problem statement (Chi & Glaser, 1985). Also, ill-structured 
problems are unstable and different solutions may be appropriate depending on the constraints faced by the problem 
solver (Petragia, 1998). 

Theories of Teaching and Learning for Problem-Solving Skills 

Several researchers suggest that constructivist and situated cognition theories of learning are effective 
approaches to solving ill-structured problems. Jonassen (1997) claims that problem solving has moved slowly away 
from information-processing theory as its conceptual base for the past two decades. He claims that traditional 
methods of problem-solving did not transfer to solving the problems situated in the real context. Also, Voss (1988) 
posits that it is necessary to develop an authentic task environment to teach ill-structured problem-solving skills 
because they are more context-dependent than well-stmctured problems. 

Jonassen (1997) suggests an instructional design model for solving ill-structured problems situated in the 
real world based on constructivist and situated cognition approaches to learning. In his design model, the learners 
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frame the design problem, recognize the divergent perspectives, collect evidence to support or reject the alternative 
proposals and ultimately synthesize their own understanding of the situation rather than find a solution for a 
prescribed problem, wherein learners must monitor the performance of the chosen solution and adapt it through an 
iterative process. 

The instructional model that is centered on authentic contexts can be an appropriate approach to providing 
the learner with a meaningful and authentic learning environment for problem solving. In order to provide authentic 
learning environments, it is important to design learning environments which support; (1) the construction of 
knowledge based on internal and social negotiation, articulation and reflection, (2) a meaningful and authentic 
context for learning and using the constructed knowledge, and (3) collaboration among learners and with the teacher, 
whose role is to coach or mentor the learners’ construction of knowledge (Jonassen, 1994). 

Constructivists also emphasize learning through context -rich, experience-based activities for the learner’ 
construction of knowledge (Duffy & Jonassen, 1992). To make learning more authentic, learners need to be 
presented with thick problems, which are rich in information, similar to what we can encounter in everyday problems. 
Two examples of learning approaches in which thick problems are presented are anchored instruction and goal-based 
scenarios (Petragia, 1998). Anchored instruction is a teaching technique to provide the learner with authentic 
problem-solving environments using multimedia technology (CTGV, 1990). A goal-based scenario is a learning-by- 
doing simulation in which students pursue a goal by practicing and using relevant knowledge and skills to achieve 
their goal (Campbell & Monson, 1994). 

The constructivist approach to designing instruction focuses more on designing learning environments that 
foster the construction of knowledge on the part of the learner rather than controlling its process by sequencing the 
information presented to the learner. Jonassen (1994) suggests an instructional design model from the constructivist 
perspective. In his design model, he suggests a learning environment which supports the construction of 
knowledge, a meaningful, authentic contexts, and collaboration with the instructor and among the learner. In such a 
learning environment, the instructor plays a role of coach or mentor. Similarly, Savery and Duffy (1995) suggest 
theoretical principles for problem-based learning as follows: 

• Learning should be relevant 

• Instructional goals should be consistent with the learner’s goals 

• Cognitive demands and tasks in the learning environment should be consistent with cognitive demands and 

tasks for which the learner is being prepared 

• Teacher’s role is to challenge student’s thinking 

• Students’ ideas should be tested against alternate views through social negotiation and collaborative 

learning groups 

• Encourage reflection on the learning process (p.l37). 

Animated Pedagogical Agents 

An animated pedagogical agent is a computer-mediated character that provides social interactions with 
students to create rich, knowledge-based learning environments for the learner (Johnson, Rickel, & Lester, 2000). 
Animated pedagogical agents can provide such human-computer interaction as interactive demonstrations, 
navigational guidance, attention guides (gaze and gesture), nonverbal feedback (e.g. a nod of approval, shaking head 
to indicate disapproval), conversational signals, conveying and eliciting emotion, and virtual teammates (Johnson, 
Rickel, & Lester, 2000). 

In a pedagogical sense, animated pedagogical agents provide learners with “scaffolding in the form of 
highly contextualized problem-solving advice that is customized to each learner’’ (Moreno, Mayer, & Lester, 2000). 
Research indicates that animated pedagogical agents increase the learner’s motivation to engage in learning activities 
in the online environment. Learners also pay higher attention to the task than they do without such an agent 
(Johnson, Rickel, & Lester, 2000). Also, Moreno and et al. (2000) have found that some aspects of animated 
pedagogical agents (e.g. auditory presence, dialogic language style) help students learn in the context of social 
interactions with the animated social agent produce better learning outcomes than the students who learn in a text - 
based computer context without such a presence. 

Methods 
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In this section, the ways to support and facilitate personalized learning for adult learners in online 
environments are discussed. The methods are broken into four components and each component is detailed below. 

1. Goal -Setting Activities 

In this first step of the instruction, the learner chooses a set of subjects that matches his/her learning needs. 
To do so, the learner’s prior knowledge and experience level will be measured to provide prerequisite instructions and 
to gauge the appropriate level of authenticity of the learning activities for the individual learner. Also, the learner will 
set his/her own expectations and goals of his/her learning, which will result in individualized learning objectives and 
methods to produce learning outcomes that meet individual needs. More detailed procedures of the goal-setting 
activities are as follows. 

(1) The learner chooses the subject areas of his/her interest and needs. 

• The animated pedagogical agent (henceforth, agent) provides the learner with a list of learning 
domains. The learner can choose learning domains that match his/her problem areas (e.g. economy, 
management, marketing, etc.). 

• Since real-world problems often contain learning domains from multiple disciplines, more than one 
learning domain can be selected. 

(2) Measure the learner’s prior knowledge and experience level. 

• The purpose of measuring the learner’s prior knowledge is to determine the level of complexity in 
the task that s/he will be engaged in and to provide supplemental instruction to the learner as 
needed before s/he begins working on the task. 

• A pretest is given to assess the learner’s entry knowledge or the learner can rate his/her own 
knowledge/skill level. 

• In addition to the learner’s prior knowledge, the learner also rates his/her experience level in the 
subject in order to determine the level of complexity in his/her learning task that s/he will be 
engaged in. 

• For the learner in the training setting, the peer and supervisor’s feedback on their performance can 
be also taken into account for measuring the learner’s entry level. 

(3) The learner sets his/her expectations and the goals of learning with guidance from instructor. 

• To set the learning goals that meet individual learning needs and also to guide the learner into self- 
regulated learning, the learner sets his/her own learning goal with guidance from the instructor. 

• The learner posts his/her expectations and goals of the instruction at the LMS and the instructor 
gives feedback on it regarding its clarity, plausibility, and the appropriateness of the anticipated 
learning outcomes. 

• To give guidance on how to write goals and expectations to the learner, the instructor posts some 
examples of such statements on the LMS. 

• The learner sets criteria that deem meaningful to him/her to evaluate his/her learning to facilitate 
self-regulated learning (Como & Randi, 1999). 

2. Engaging in the Learning Task 

In this step, the learner interacts with the agent to choose a task that s/he will be engaged in throughout the 
learning process in a problem-centered learning approach. Also, the learner acquires necessary prerequisite 
knowledge/skills and is given resources needed to perform the task in a just-in-time fashion. 

(1) The learner chooses the context s/he is interested in for his/her learning task. 

• The agent presents the learner a variety of contexts that the learner can choose from (e.g. business, 
education, technical, ...) 

• The learner selects a context that s/he wants to be engaged in based on his/her own learning need 
and interest. 

(2) The learner selects a learning task in the context of his/her choice. 

• The agent presents multiple forms of tasks that the learner can choose from (e.g. scenarios, cases, 
projects, etc.). 

• The tasks are provided in the context of his/her choice. 

• The tasks should be authentic that address real world issues. 
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• The level of complexity of the task should be suited for the learner’ s entry level, as 
identified in the previous step. It should also be novel enough to represent a real-world 
situation. 

• The learner selects a task of his/her choice from the list of tasks presented by the agent, 
o The expected duration of each task is given to help the learner’s choice of the task, 
o The learner chooses the format of the task based on his/her preference. 

o The learner can control the expected duration of the learning at his/her pace. 

• The learner can choose the format of the learning output (e.g., project report, presentation, action 
plans) by selecting a form of task, which leads to creating a product that s/he can use in their real 
life or work settings. 

(3) For the concepts or skills the learner needs to acquire as the prerequisites for performing the task, 
relevant modules are selected from the learning objects and are presented to the learner. 

• The agent presents the prerequisites based on the results of the pretest conducted at the goal- 
setting phase. 

• The modules are chosen from the learning objects. 

• The learner learns the prerequisites through tutorial using a generality-examples -practice with 
feedback approach. 

• When the learner explores the tutorial at his/her own pace and when s/he reaches the mastery level, 
the tutoring system guides him/her to move on to the task. 

• The instructor answers any questions that the learner encounters while learning through learning 
objects. 

(4) The learner is given additional resources on the task to be performed. 

• Use visualization tools (e.g., video, charts, graphs) to present the task (Jonassen, 1999). 

• Give worked examples to present a model for the learning (Jonassen, 1999). 

• The instructor is available as a facilitator to provide further information on how to perform the task. 

• Provide the learner with resources for exploring relevant information in a just- time-time fashion, 
o All the necessary resources should be readily accessible via online for just-in-time access, 
o The resources need to provide rich information to support the learner who copes with 

authentic problems. 

o Use a variety of formats to present the information (text, graphics, animations, etc.). 

3. Performing the Task 

In this step, the individual learner engages in the task with assistance from the agent. The instructor also 
provides guidance to promote the constructivist learning environment. 

(1) The learner performs the task exploring the information resources described earlier. 

(2) The agent guides the learner to perform the task. 

• The agent provides templates that the learner can use to perform the task. 

• It also guides the learner to the module in the learning objects system when the learner has 
questions about a subject. 

(3) Online communication with the instructor to foster the learner’s cognitive development. 

• The instructor can use a variety of modeling, coaching and scaffolding techniques, as suggested 
by Jonassen (1999), to promote a constructivist learning environment. Those techniques are as 
follows; 

o Modeling: model performance, articulate reasoning 

o Coaching: provide motivational prompts, monitor and regulate the learner’s performance, 
provoke reflection, perturb learner’ s models 

o Scaffolding: adjust task difficulty, restructure a task to supplant knowledge, provide 
alternative assessments (pp. 23 1-236) 

• Also, the instructor guides the learner on the process of his/her learning (e.g., what needs to be 
done and how s/he can do it). 

• The instructor provides the list of online learning communities that the learner can participate to 
discuss with others on the problems/tasks s/he is engaged in. 
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(4) Reflection and feedback on the process and the output of learning 

• While working on the task, the learner reflects on what worked and what didn’t work in his/her 
approach to solving the problem and posts it regularly on the LMS. 

• The instructor gives prompt feedback on the learner’s reflection and provides appropriate coaching 
and scaffolding to the learner. 

4. Evaluating and Reflecting on the Task 

Instructor’s evaluation, reflective self-evaluation, and peer evaluation are conducted to evaluate the result 
of learning. 

• The learner posts the output of his/her learning task on the LMS. 

• The learner also posts his/her reflection on the learning process. 

• The instructor and the learner evaluate the output and the process of learning according to the preset 

evaluation criteria. 

• The output is also posted in the learning community in which the learner’s has been engaged and receives 

feedback from the participants. 

Conclusions 

In conclusion, the learning-focused paradigms in this instructional theory are discussed. Also, 
considerations in applying and improving the theory are discussed in what follows. 

New Paradigms in This Theory 

Some learning-focused paradigms are adopted in this instructional theory. Those new paradigms are the 
instructor’s role as a guide of learning, the incorporation of technology to provide individualized learning, and the 
use of a learning management system to foster self-regulated learning. First, in this instructional theory, the learner 
takes initiative for the whole learning process and the instructor plays the role of guide or mentor - e.g., giving 
guidance to the learner to help him/her become a self-regulated learner. Second, this theory of instruction guides the 
learner to set individualized learning goals and instructional methods using computer technology (e.g. animated 
pedagogical agent, learning objects system, and learning management system). Finally, most learning management 
systems in current Web-based instruction are used to control the learner - e.g., to make sure the learner is doing 
his/her assignment on time - not to facilitate learning. The LMS in this instruction is used as a tool to foster the 
learner’s self-regulated learning and communication with the instructor rather than controlling their learning. 

Considerations in Implementing the Theory 

It is important to consider the limitations of an instructional theory in terms of under what circumstances it 
can or cannot work. As mentioned earlier, this instructional theory is developed for teaching higher-order thinking 
skills in ill-defined domains; hence, the methods in this instructional theory are intended to require a lot of effort from 
the learners to set their own learning goals and plan on their own learning process to facilitate highly complex 
thinking skills. However, applying this approach to teaching cognitive skills in relatively well-defined domains, for 
which the learning goals and methods can be defined more easily, might cause too much exertion of the learner’s time 
and effort. Thus, it is suggested that this instructional theory might not be an efficient approach to teaching 
cognitive skills in well-defined domains. 

In addition, particular attention needs to be paid to the instructor’s role in this individualized learning 
environment. The instructor needs to play the role of a guide or facilitator of the learning environment in which the 
learner is provided with feedback and guidance on the learning process based on their personal learning needs. For 
this, the instructor can provide information on the learning resources and provide prompt feedback on the learner 
output in each step of the learning process. Most of all, guidance to the learner for him/her to become a self-directed 
learner is warranted. One way to achieve this goal is to help the learner reflect on his/her learning process. 

Evaluating and Improving the Theory 

You will need to evaluate the plausibility of this instructional theory in the actual context that you will be 
applying it to in order to make sure this theory works well in your situation. Reigeluth (1983, 1999) has suggested 
that an instructional theory can be evaluated based on the criteria of effectiveness, efficiency and appeal of an 
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instruction designed from the theory. Effectiveness is the extent to which the application of the theory attains the 
goal in a given situation. Efficiency is the extent to which resources (costs in terms of time, money, etc.) are warranted 
for theory implementation. Appeal refers to how enjoyable the resulting instructions are designed from the theory. 
Erick and Reigeluth (1999) also suggest that these three criteria may be valued differently in different situations 
according to the needs or wants of the stakeholders. Therefore, you need to consider what criteria are more 
meaningful and critical for your learners when you evaluate this instructional theory. 

Also, no instructional theory is perfect, as the main concern for developing and testing an instructional 
theory is preferability - i.e., does this method attain the goals in a given situation than any others? - rather than 
validity (Reigeluth & Erick, 1999). Therefore, it needs to go through formative research for it to become more 
preferable. The underling logic or purpose of conducting formative research on an instructional theory is to apply 
the instructional theory in a real instructional setting and identify weaknesses found in the application of the theory 
which may reflect ways to improve it (Reigeluth & Erick, 1999). The methodological procedures for conducting 
formative research are described by Reigeluth and Erick (1999). 

Since the instructional theory presented in this paper has not gone any field testing nor has been studied 
for its weaknesses, formative research is highly recommended to improve this theory so that it can better serve the 
purpose of the theory and attain desired outcomes. 
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Adding Interactivity to Neuroanatomy Slide Images Using Layers, Image Maps, and 

JavaScript Behaviors 

Taeyeol Park 

Georgetown University Medical Center 
Abstract 

The Department of Neuroscience and the Faculty and Curriculum Support (FACS) Center at the Georgetown 
University Medical Center developed an online interactive learning tool to help students understand brain 
structures and memorize their elements with neuroanatomy slide collections. Each slide page was designed to 
show students a key when they move a cursor over a particular region of an image. The interactive learning tool 
was created with Macromedia Dreamweaver by using its layers, image maps, and behaviors features that are easy 
to use and learn. The learning tool for neuroanatomy slide collections product was launched and overwhelmingly 
successful with the faculty and students. 

Background 

The Faculty and Curriculum Support (FACS) Center of the Dahlgren Memorial Library and the Department 
of Neuroscience at the Georgetown University Medical Center previously provided first year medical students with 
35mm slides and web based slide collections as an education resource to conplement the neuroscience course 
requirement. Most of the scanned slide images and text for describing the images were static web pages. The web 
based slide collections did not have any interactivity and were not complete since some were still in 35 mm slides. 
Although some other courses in the school have already used online digital slide collections as their course 
supplements, they were the collections of images just for delivering information without involving interactions 
between students and learning materials. 

Since neuroanatomy is best learned through repetition, students needed a tool that allows them to review 
high quality, labeled images of brain and spinal cord often and at the same time, test their knowledge of the 
neuroanatomy. Faculty in the Neuroscience department required a tool to help first year medical and physiology 
graduate students understand brain structures in relation to pathways and function and memorize their elements with 
labels by using digitized human anatomical specimens. They wanted a learning tool that was interactive as well as 
online. 
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Figure 1 . The Neuroscience Lab pages 



The project focused on two main issues, pedagogy and technology. The pedagogical issue was how to 
create the interactive learning tool with the slide images. Although some other courses have already used online 
digital slide collections as their course supplements, they were the collections of images just for delivering 
information without involving a student’s interaction except navigational activities. The project was to create a tool 
more than just for collecting and presenting information online. It wanted a learning tool to let students relate keys to 
areas in a slide image through their interactions with the graphic image. 

The technological issue was how to create the tool with a limited level of technology that does not require 
advanced skills or complicated methods so that a non-technician can easily expand its contents and maintain it. For 
this purpose, the instructional technology support staff of the FACS center selected Dreamweaver and used its built- 
in functions for making layers, image maps, and behaviors to create the learning interactions with graphical interface. 

Method 

Based on the notion that instructional materials with higher interaction level lead to more engagement and 
elaboration of learning, the FACS center developed a tool more than just for collecting and presenting information 
online. The online interactive learning tool for neuroscience slide collections project was designed to help students 
understand brain structure elements and memorize their labels and locations in the slide images through the 
interactions with the images. Each slide page was designed to show students a key (anatomical label) when they 
move a cursor over a particular region of an image. This new interactive learning tool was created with Macromedia 
Dreamweaver 4.0 by using its layers, image maps, and behaviors (events and actions) features. 




Figure 2. Creating an image map for a target area and a layer for displaying text 



The project’s second goal was to create the tool with a limited level of technology that does not require 
advanced skills or complicated methods so that a non-technician can easily expand its contents and maintain it. To 
achieve the goal, the FACS staff selected Dreamweaver, a web authoring application, and used its built-in functions 
for making layers, image maps, and JavaScript behaviors to create learning interactions and graphical interface. The 
techniques were so easy that the department staff who had no background in web page design could learn them 
without any difficulty after a one-on-one training session. 

The department staff scanned the slide images and put all labels on the images using layers in Adobe 
Photoshop. After the FACS staff designed a frame-based main web page for the slide images and learning 
interactions as a template, the department staff completed all the page designs using the template. The FACS staff 
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created a typical page for a slide image and a web page frame structure as a template including all the learning 
interactions. The department staff scanned the slide images and put all labels for the keys on the slide images using 
layers in Photoshop. The department staff completed all the web page design for the interactive learning using the 
template with her skills trained by the FACS staff. 
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Figure 3. Assigning behavior (events and actions) to an image map 



Evaluation 

The online interactive learning tool for neuroanatomy slide collections product was launched and 
overwhelmingly successful with the faculty and students. After the initial trial of the site in a Neuroscience class, the 
department received several very good reviews of the product and is expected to expand the product in the next 
semester. The FACS center has received qualitative analysis data from instructors of the course. 

Conclusion 

All reactions to the site from both students and instructors have been positive. The students are finding 
the images to be very helpful for studying and request that we expand the site to encompass more materials that is 
used in the class. The project prototypes were also valuable resources and could be used as templates for future 
related educational projects. They could be extensively applied for other courses dealing with graphic images and 
learning interactions using web and multimedia. The project became an excellent example of interdepartmental 
collaborative work in the online multimedia course material development in a medical school curriculum. 
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Diversity Training for the Hospitality Industry: An investigation on the effectiveness 
of applying structured-writing principles to facilitate diversity training. 

Doris Choy 
Purdue University 
Tristan Johnson 
Learning Systems Institute 
Florida State University 
Abstract 

Hospitality industry needs high quality diversity training materials that could explicate the complicated diversity 
issues and educate the employees effectively because of the diverse nature of the industry. Structured-writing 
principles are known for analyzing complicated information into organized content that facilitate learning. 
Structured-writing principles are comprised of techniques for formulating, designing, and writing text to improve 
comprehension and retrieval.The purpose of the study is to investigate if the implementation of structured-writing 
principles to the development of effective diversity training materials would create high quality materials that 
improve learners ' performance and thus is beneficial to the industry. 

Introduction 

Diversity training is becoming increasingly important in United States as the population is becoming more 
diverse. By the year 2050, according to LaVecchia (1998), half of the population in the United States will consistof 
people that are currently considered as “minorities.” Moreover, the aging of the population in general create new 
challenges in how to utilize the aging population as well as how to market to this audience effectively. Like many 
other industries, the hospitality industry started to pay attention to various diversity issues since the creation of the 
Equal Employment Opportunity Commission (EEOC) in 1965 (EEOC, 2001). The mission of the EEOC is to eradicate 
employment discrimination at workplace so that individuals can be hired, promoted, and compensated based on their 
abilities and qualifications. 

Review of Literature 

Overview of the importance of diversity training for the hospitality industry 

Within the EEOC, major federal statutes that affect the hospitality industry included: the Equal Pay Act of 
1963, which prohibit sex-based wage discrimination between men and women in the same organization with similar job 
responsibilities. Title VII of the Civil Rights Act of 1964, which prohibits unfair employment discrimination based on 
race, color, sex, religion, and nation origin (Woods, 1997). The Civil Rights Act of 1991 provided guidelines on 
penalties in cases of intentional employment discrimination and clarified the legislations related to disparate impact 
("The Civil Rights Act of 1991,” 1991). The Age Discrimination in Employment Act of 1967, which prohibits 
employment discrimination against employees and applicants who are older than 40 years of age. As the population 
is aging, this problem becomes more apparent. Age discrimination affects the economy as increasing number of older 
workers are unemployed and thus unable to live with economic independence ("The Age Discrimination in 
Employment Act of 1967,” 1967). The Americans with Disabilities Act of 1990 (ADA) prohibits employment 
discrimination against qualified individuals with disabilities (Woods & Kavanaugh, 1992). Diversity issues are 
complicated, difficult to understand, and many hospitality operators feel that the federal statutes are tricky 
(Nolo.com, 2001). Therefore, it is necessary to develop effective diversity trainings for the employers and the 
management of the hospitality industry in order to educate their employees and comply with the legislations. 
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However, diversity issues are not limited to complying with the legislations anymore due to the constant 
interactions between the diverse workforce, the international customers and multinational corporations.. In the 
service-oriented hospitality industry, potential problems such as workplace hostility, negative customer relations and 
bad reputation of the industry would arise if efficient and effective diversity trainings are not provided to the 
employees. Diversity training becomes essential for the productivity, competitiveness, harmony, and the success of 
the hospitality industry (Lee & Chon, 2000). 

Diversity in human resources translates into providing equal opportunities to employees in recruiting, 
training, promotion and compensation. In order to create the openness at the workplace and promote diversity, it 
becomes an organization’s responsibility to demonstrate their respects on individual’s differences and encourage all 
employees to develop themselves in the organization (Groschl & Doherty, 1999). While many diversity training 
programs addressed issues such as increase in sensitivity and appreciate cultural differences, other important issues 
such as developing technical skills in dealing with diversity and facilitating cross-cultural skills were not addressed in 
many diversity trainings (Lee & Chon, 2000). Effective diversity training should include: clear training objective, 
follow-up assessment of trainees in how the training impact them and evaluation in how the trainees apply their 
diversity training in their workplace (Cox, 1996). 

The purchasing power of minority groups is increasingly at a fast pace. It is projected that African 
Americans’ purchasing power will rise 72 percent in this decade, whereas the purchasing power of the general 
population is projected to be 56 percent (Whitford & Higley, 1998). Therefore, it is vital to provide diversity trainings 
to employees about the importance of providing inclusive and high quality services to all customers. Minority 
markets such as African Americans and Asian Americans are having more purchasing power (Nigro, 1998) and 
marketing effectively to the specific customers can have a significant impact to the success of the hospitality 
industry. 

The ultimate goal of effective diversity training is to help your employees to develop a sense of common 
goals of serving customers and providing high quality of services (Rousseau, 1997). While the differences among 
employees and customers are being addressed, employees should understand that all of them work as a team towards 
a common goal. Their differences and uniqueness are essential for the success of the organization. Moreover, owners 
and managers need to continuously emphasize the importance of common goals and help employees to develop the 
“diversity lens’’, where everyone in the organization will view everything with the “diversity lens’’. In the long run, 
the organization will be able to foster a mutual respect environment. This can only be achieved if the work 
environment and employees at all levels respect the differences among co-workers and value each others’ 
uniqueness. 

As a result, one can understand that diversity training consists of complicated information such as different 
statutes and legislations; perception changes of the employees related to interacting with different people; personal 
development within employees; maintaining a positive work environment that focus on a team’s common goal as well 
as individual’s uniqueness. Therefore, it is not difficult to understand why the hospitality industry had not yet 
developed some high quality diversity trainings for their employees. 

Overview of Structured-Writing Principles 

In order to develop effective diversity training materials for the hospitality industry, structured-writing 
principles are selected to be implemented to the development of diversity training materials. Structured-writing 
principles have been implemented to various instructional materials. It is believed that structured-writing would be 
able to improve the quality of diversity training materials and thus increase the effectiveness of the material. 

Locating and retrieving information effectively becomes a challenging task with all the information available 
to people. In the past, people worried that they cannot receive enough information. Nowadays, people worry about 
being flooded with unlimited information and are having difficulties in sorting through all of them effectively. 
However, few researchers called attention to the different writing and reading requirements, even though the world of 
business and technology is growing so complex and information is changing faster than we could learn it (Horn, 

1993). 

Structured-writing has been developed as one of the instructional strategies to address many of the 
perennial problems most people have when working on a complex written communication task (Horn, 1998). Many 
terms are used to describe the techniques employed in text design. One that is used often is “Information Mapping’’, 
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which was developed by Horn. The phrase “Information Mapping’’ is a registered trademark of Information Mapping, 
Inc. Several similar terms have been developed such as “information simplification’’, “mapping”, “message design,” 
and ‘structured-writing.” 

Structured-writing seems to describe most accurately the techniques for formulating, designing, and writing 
text to improve comprehension and retrieval. The techniques chunk information into domains called information 
blocks, which can precisely describe complex information (Horn, 1999). Each information block is a small chunk of 
information and consists of similar content with strong relationships with each other. Structured-writing is a tool for 
writers, instructional designers, teachers, and anyone who try to disseminate information to effectively organize and 
visually present almost any type of raw information so readers can grasp its significance more quickly (Whiteside & 
Whiteside, 1993). Balan (1989) suggested that by carefully structuring text and applying typographical cues, the 
learner spend less time processing text and more time learning new material. Hence, it improves the instructional value 
of any instructional material. 

What is Structured-Writing? Principles of Structure -Writing Strategies 

Structured-writing can be defined as the strategies for formulating, designing, and writing text to improve 
comprehension and retrieval of information by the readers (Whiteside & Whiteside, 1993). It provides a systematic 
way to analyze and design information in written documents. It was first developed in trying to make reading and 
learning easier and quicker for readers in a complex information rich environment. There are four major principles in 
structured-writing, they include: information blocks, information maps, labeling, and formatting. 

Information Blocks 

First, the content is analyzed thoroughly to discover the underlying structure of the subject matter. Content 
analysis helps to specify what key information is missing and help to simplify the following content development 
stages as writers can identify the missing pieces early. The content of the subject matter is broken down into the 
smallest practical unit of meaning for writing documents and formed the basic units. These units are essential to the 
ability in teaching others with great uniformity to sort sentences of the subject matter into an easily understood 
taxonomy (Horn, 1992). Thus, the content is being categorized into basic units called “information blocks” according 
to the underlying structure through the process of “chunking”. The four basic principles of information blocks are 
chunking — group all information into small units; labeling — name the blocks according to the content and key 
concepts; relevance — keep one key point within each block; and consistency — use similar wordings, 
terminologies, formats, and organizations (Horn, 1998). 

Information Maps 

A large number of information blocks gathered without organization would fail to provide readers with 
logical ways to cluster important concepts, procedures (Horn, 1993), and hinder the readers’ abilities to search or 
retrieve information from the content. In order to solve this problem, a number of information blocks would be 
arranged into an “information map”. An information map consists of one to nine information blocks with related 
content. It provides an important intermediate level of specification (Horn, 1993) of the content. All the blocks related 
to similar key points could be clustered into information map and label appropriately. 

Information has a topography like geographical terrain with peaks and valleys, cities and countryside, and 
roads to connect them (Horn, 1992). Information maps should act like a geographical map that guides the readers to 
read, search, and understand the content of the subject matter and bring the readers to their desired destinations 
effectively and efficiently. 

Labeling 

During the chunking process, the content has been formulated into information blocks. Each block has been 
labeled appropriately based on the key points. However, after the information blocks are organized into information 
maps, additional labeling is required to represent the main ideas clearly. 

In order to distinguish the differences in labeling information maps and information blocks, the headings of 
information maps are called “map titles” and the labels of information blocks are called “block labels”. The purpose of 
both types of labels is to help describe the specific content of the subject matter, the functions of the block or the 
map, and combinations of these two types (Horn, 1993). The importance of labeling enables the readers to read the 
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content of the subject matter quickly and understand the structure of the document; to locate the important points 
that readers are looking for; to manage information gathering and analysis; and to retrieve information from the 
content efficiently. 

Consistent Formatting 

Consistent use of information blocks, block labels, and map titles in the formatting of structured-writing 
enhance readers’ ability in scanning documents efficiently. Readers will be able to locate the content easily, scan the 
important information and skip the details when time is limited. While the formats were initially developed to aid the 
reference and rapid scanning modes of use, these similar formats also aid learners to pre -organize and size up their 
learning tasks during the learning process (Horn, 1993). 

Consistent map titles provide essential information to orient the readers while block labels provide basic 
description of the information blocks. Readers immediately recognize how easy it is to scan and skip to what they 
want to read. They recognized the rapid access that the formats provide on every page. Therefore, they begin to have 
confidence in the document and in their ability to understand what it contains (Horn, 1993). Implementing structured- 
writing principles to diversity training will be helpful because complicated diversity issues can be broken down into 
information blocks and then being constructed into information maps that would be easy to scan and understanding, 
which would improve the instructional value of the diversity training material. 

Advantages of structured-writing included but not limited to: easy-to-scan; better information retrieval and 
understanding; effective writing techniques; decrease in reading time and the difficulties; reduction in time needed to 
prepare textual materials; and assist in the development of just-in-time training. 

Structured-writing strategies consist of strengths that could enhance learning in an educational 
environment. However, as the concept was originally developed for the business and industry, additional studies are 
necessary to investigate the pros and cons in implementing structured-writing strategies in classrooms and also what 
kind of alterations could be done to make it a better fit for teachers as well as students. 

The simplicity and clarity in structured-writing principles is beneficial to the development of diversity 
training because it is a complex subject matter. In general, hospitality employees perceived diversity training as 
confusing and difficult to understand. Moreover, many of the employees of the industry do not have high education 
level or reading abilities. Thus, presenting information clearly and in an easy to read format is essential in motivating 
the employees to learn. As a result, implementing structured-writing principles to diversity training material could be 
a potential solution to solve the acute need of high quality effective diversity training material of the hospitality 
industry. 

Purpose of the Study 

The goal of this research study was to implement structured-writing principles in the development of 
efficient and effective diversity training materials for hospitality front line employees. The purpose of the study is to 
determine if the application of structured-writing principles to the developmental process of diversity training 
materials for the hospitality employees facilitated training and improved learners’ performance in recalling the 
information. As a result, it is hypothesized that structured writing principles would improve the learners recall abilities 
when they learn about basic diversity issues. 

Research Questions 

Based on the purpose of the study, a posttest only experimental design was developed to answer the 
following research questions: 

• Question 1: Are structured-writing principles an effective instructional strategy for diversity training? 

• Question 2: Can the implementation of structured-writing principles to diversity training materials for 
the hospitality industry improve learners’ learning when compared to traditional prose-writing format? 

• Question 3: Can structured-writing principles improve the learners’ recall abilities in diversity training? 

Methodology 

In order to solve the problem, the researcher used an approach that has been shown effective in other areas 
to analyze and organize many perennial problems — structured-writing (Horn, 1993). The goal of the study was to 
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implement structured-writing principles in order to design and develop efficient and effective diversity training 
materials for hospitality hourly employees. 

There are three hypotheses in this research study: 

HI: The participants who received the original structurally written training material (SW 1) will perform better than 
those who received prose written training material (P). 

H2: The participants who received the revised form of stmcturally written training material (SW 2) will perform better 
than those who received the original structurally written training material (SW 1). 

H3: The participants who received the revised form of stmcturally written training material (SW 2) will perform better 
than those who received prose written training material (P). 

The prose written material (P), original stmcturally written material (SW 1) and the revised version of 
structurally written training material (SW 2) will be discussed in detail in the procedure section. 

Sample/ sampling method 

The participants of the study were undergraduate students in an introductory course of the Department of 
Hospitality and Tourism Management. This is the largest course offered by the department once a year. The average 
enrollment of the course is about 200 students. According to previous enrollment data, a vast majority of the 
students in this course are in their freshmen or sophomore years. Their demographics are similar to the hourly 
employees in the hospitality industry. In fact, many of them work at local restaurants and hotels as hourly employees. 
These participants were chosen to represent the hospitality hourly employees because they have diverse 
backgrounds and limited knowledge about diversity, which are similar to hospitality hourly employees. The class was 
randomly divided into three groups. Each group received different formats of training materials and the same posttest 
after they learned from the materials. 

Procedure 

There were two main stages in the data collecting process: the developmental stage and the implementation 
stage. The developmental stage mainly focuses on the design and development of the instructional materials and 
posttest. The implementation stage focuses on data collection of the participants. 

Developmental stage 

Stage One: The development of prose-written material — traditional prose- written instructional material was 
developed by reviewing textbooks related to diversity training. Sections in different textbooks were selected as the 
foundation to create the prose-written material. This was the control in the study — the traditional prose-written 
material. 

Stage Two: The development of the original version structurally written material (SW 1) — the content in 
the prose-written material was analyzed thoroughly to develop the original version of the structurally written material. 
It was created based on the principles of structured-writing. As a result of these principles, information blocks 
become far more usable text to scan, to read and to memorize (Horn, 1998). 

Stage Three: The development of revised version structurally written material (SW 2) — the revised version 
of structurally written material was developed by incorporating students’ feedbacks in how to improve the original 
version of structurally written material. As Horn (1993) pointed out, structured-writing principles were developed for 
effective business communications. Some of the structured-writing principles may not be as valuable in classroom 
settings. Therefore, a preliminary study was conducted with a small number of undergraduate students (about 15) 
from Hospitality and Tourism Management. The students were asked to review the original version of the structurally 
written material and take the posttest. After that, the students were asked to fill out a questionnaire consisting of 
open-ended questions such as, “how do you think the training material could be change to help you learn the 
content?” Changes were made to the original structurally written material based on their recommendations and the 
revised version of structurally written material was developed. 

The posttest material was developed based on the content of the diversity training materials in the prose 
written material and the original structurally written material. Since the content was consistent across different 
versions of instructional materials, it was not a major concern that the test was being developed prior to the 
completion of all instmctional materials. 
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Figure 1: Developmental stage of prose written material, original structurally written material, revised version of 
structurally written material, and posttest. 

Implementation Stage 

This is the actual treatment and test administration stage. As there are a large number of participants, it was 
crucial to plan the process carefully in order to collect the data efficiently. The participants were randomly assigned 
to the three treatments: prose- written (P), original structurally written (SW 1) and revised structurally written materials 
(SW 2). Participants received one of the instructional materials to learn about the diversity-related content. They were 
given 20 minutes to review the materials. Then, they returned the instructional materials and received the posttest. 
They had about 20 minutes to complete the posttest. The researcher collected the posttests at the end of the limited 
time period. 




Figure 2: Implementation stage of the data collection process 
Research design — Posttest only experimental design 

The study was a posttest only experimental design. The experiment was administered without a pretest 
because the participants were randomly assigned to three different control and treatment groups. Therefore, the 
treatment and control groups were not systematically different from each other. As a large number of participants 
were available, random assignment was effective in dividing them into equivalent groups. Thus, systematic 
differences or biases in assigning participants to the control and treatment groups were minimized. As there were no 
systematic differences, there was no need to conduct a pretest. An additional advantage of not conducting a pretest 
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was that there was no need to worry about testing effect where the participants may learn about the test in the 
pretest and therefore able to perform better in the posttest. 

Methods of data analysis 

One-way Analysis of Variance was used to analyze the data because the participants were divided into more 
than two groups. After that, the Tukey’s Honestly Significant Difference test was used to determine the significant 
differences among the individual group means. 

ANOVA was used to analyze the data among the three groups on their posttest scores, which measured 
their abilities in understanding and recalling the content of the instructional materials. Statistical analysis calculated 
the F ratio of the participants’ posttest scores in different treatment groups. The F ratio was able to see if the 
differences between the three treatment groups were significantly larger than the variations within individual groups. 

The statistical analysis of ANOVA will only produce an F ratio claiming there are significant differences 
among the three groups. However, F ratio cannot explain which group is significantly different from what other 
groups. Therefore, after conducting ANOVA, Tukey’s Honestly Significant Difference test was performed to analyze 
which individual group was significantly different from which group. 

The purpose of the study was to investigate the effectiveness of implementing structured-writing principles 
to diversity training materials for hospitality hourly employees. The results of this research will benefit future 
instructional designers by increasing their understandings of structured-writing principles and how to effectively 
implement the principles in future instructional development projects. 

Preliminary Data Results 

A pilot study was conducted in the summer session prior to the full scale data collection. The results of the 
pilot study showed that participants learned from the original version structurally written material (SW 1) group 
performed significantly better that participants learned from the prose written training material (P). Their means are 
29.375 and 19.75 respectively (p-value = 0.02). The Cronbach alpha was estimated by using the data of the pilot study 
to examine the reliability of the testing instrument. The preliminary Cronbach alpha was 0.89. After deleting two items 
from the posttest, the reliability was boosted to 0.90. Therefore, the pilot data showed that the posttest is a reliable 
testing instrument to be used for the study. 

Discussions 

Structured-writing principles have been applied to learning in many different domains. However, there are no 
previous research studies that attempt to apply structured-writing principles to the design and development of 
diversity training for the hospitality industry. Therefore, this study adopted the structured-writing principles to 
develop diversity training materials that could facilitate learners in understanding diversity. By utilizing various 
structured-writing principles, the instructional designers were able to effectively organize, visually present 
(Whiteside & Whiteside, 1993) and simplify the complex diversity issues into usable information for the hourly 
hospitality employees. The learners are able to learn about complicated diversity issues and the implementation of 
structured-writing facilitates their recall abilities. This paper presented some preliminary results of the study which 
showed significant differences between the performances of participants from the two groups. Therefore, it is 
projected that the results of the full scale study would be similar to the results of the preliminary data collection. 

Future research studies can try to investigate the effectiveness of implementing structured-writing 
principles into different content areas. In this study, structured-writing was implemented on paper-based 
instructional materials. As computer-assisted instructions are increasingly popular, researchers can also try to 
examine the possibility and the effectiveness of implementing structured-writing principles in computer-based textual 
information. 



Conclusion 

The purpose of the study is to examine the implementation of structured-writing principles to diversity 
training materials for the hospitality employees and find out if the principles can help to facilitate training and 
improve learners’ performance in recalling the information when compare to learners who learned from traditional 
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prose written training materials. As a result, it is hypothesized that structured writing principles would improve the 
learners recall abilities when they learn about basic diversity issues. 

Structured-writing principles are originally developed to help improve the effectiveness of business 
communications in the business settings because of the large amount of information. As we are transferring more and 
more information to our learners in the instructional and educational settings, we need to look for more effective 
methods in communicating the information to the learners. Structured-writing has been widely adopted in the 
business settings for its efficiency and effectiveness. However, limited studies had been conducted in the 
instructional settings that are related to structured-writing principles. Therefore, this study is conducted to evaluate 
the possibility of implement the principles in an instructional environment. In the long run, the instructional designers 
will benefit from the results of the study as they can learn more about the effectiveness of implementing structured- 
writing principles to the development of various instructional materials. 
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Abstract 

Structured-writing strategies consisted of analyzing, organizing, sequencing, and displaying the various units of 
information. The major components of structured writing included: information blocks, information map, labeling 
and consistent formatting. Advantages of implementing structured writing strategies are easiness to scan 
information, enhance information recall and retrieval, improvement of writing techniques and applicable to 
develop effective just-in-time training. Some challenges of applying structured writing strategies would be 
difficulty to analyze the increasingly complex and rapidly changing information, resistance of readers to learn new 
reading techniques, and lack of motivational strategies in the development of structurally written document. 

Introduction 

Locating and retrieving information effectively becomes a challenging task with all the information available 
to people. In the past, people worried that they cannot receive enough information. Nowadays, people worry about 
being flooded with unlimited information and are having difficulties in sorting through all of them effectively. Even 
though the world of business and technology is growing so complex and information is changing faster than we 
could learn it, few researchers called attention to the different writing and reading strategies (Horn, 1993). 

Structured writing has been developed to address many of the perennial problems most people have when 
working on a complex written communication task (Horn, 1998). Many terms are used to describe the techniques 
employed in text design. One that is used often is “Information Mapping’’, which was developed by Horn. Other 
terms have been developed such as “information simplification’’, “mapping”, “message design,” and ‘structured- 
writing.” 

Structured-writing seems to describe most accurately the techniques for formulating, design, and writing text 
to improve comprehension and retrieval. The techniques are to chunk information into domains called information 
blocks, which can precisely describe complex information (Horn, 1999). Information blocks are small chunks of 
information that are similar in that cluster sentences have strong relationship with each other. Structured-writing is a 
tool for writers, instructional designers, teachers, and anyone who try to disseminate information. It helps to 
effectively organize and visually present almost any type of raw information so readers can grasp its significance 
more quickly (Whiteside & Whiteside, 1993). Balan (1989) suggested that by carefully structuring text and applying 
typographical cues, the learner spend less time processing text and more time learning new material. Hence, it 
improves the instructional value of any instructional material. 

Horn (1999) explained that stmctured-writing provides a systematic way of analyzing any subject matter to 
be conveyed into a written document. It consists of a set of techniques for analyzing, organizing, sequencing, and 
displaying the various units of information. Thus, the method can be said to capture and sort the “core” sentences of 
the subject matter. It clusters them into meaningful units for further refinement in the writing phase of documentation 
and training materials (Horn, 1993). The structured-writing techniques help to analyze and organize complex 
knowledge into easy-to-access information for readers, who can choose to read in detail or scan through it quickly to 
grasp the key points (Information Mapping, 2001). 

This paperpresents four major sections: the principles of structured-writing strategies; the advantages of 
structured-writing; the challenges of structured-writing; and the application of structured-writing in education. 
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Principles of Structure- Writing Strategies 

Structured-writing is defined as the techniques for formulating, designing, and writing text to improve 
comprehension and retrieval of information by the readers (Whiteside & Whiteside, 1993). It provides a systematic 
way to analyze and design information in written documents. It was first developed in trying to make reading and 
learning easier and quicker for readers in a complex information environment. There are four major principles in 
structured-writing, they include: information blocks, information maps, labeling, and consistent formatting. 

Information Blocks 

At the first stage, the content is analyzed thoroughly to discover the underlying structure of the subject 
matter. Content analysis helps to specify what key information is missing and help to simplify the content 
development stages as writers can identify the missing pieces early. The content of the subject matter is being break 
down into the smallest practical unit of meaning for writing documents and formed the basic units. These units are 
essential to the ability in teaching others easily and with great uniformity to sort sentences of the subject matter into 
an easily understood taxonomy (Horn, 1992). Thus, the content is being categorized into basic units called 
“information blocks’’ according to the underlying structure through the process of “chunking’’. 

The four basic principles of information blocks are: (1) chunking: group all information into small units; (2) 
labeling: name the blocks according to the content and key concepts; (3) relevance: keep one key point within each 
block; and (4) consistency: use similar wordings, terminologies, formats, and organizations (Horn, 1998). 

To ensure consistency in similar content, consistent wording, labels, formats, organizations, and sequences 
should be used in the block (Horn, 1998). Information in each information block should only be representing one main 
point for the relevance purpose. If there is other information that is important for the readers to know but irrelevant to 
this main point, it should be placed in another block to avoid readers’ confusion (Horn, 1998). 

Information Maps 

A large number of information blocks gathered together without organization or management would fail to 
provide readers logical ways to cluster important concepts, procedures, (Horn, 1993) and hinder the readers’ abilities 
to search or retrieve information from the content. In order to solve this problem, a number of information blocks 
would be arranged into an “information map’’. An information map consists of one to nine information blocks with 
related content. It provides an important intermediate level of specification (Horn, 1993) of the content. All the blocks 
related to similar key points could be clustered into information map and label appropriately. 

Information has a topography like geographical terrain with peaks and valleys, cities and countryside, and 
roads to connect them (Horn, 1992). Information maps should act like a geographical map that guides the readers to 
read, search, and understand the content of the subject matter and bring the readers to their desired destinations 
effectively and efficiently. 

Labeling 

During the chunking process, the content has been formulated into information blocks. Each block has been 
labeled appropriately based on the key points. However, after the information blocks are organized into information 
maps, additional labeling is required to represent the main ideas clearly. 

In order to distinguish the differences in labeling information maps and information blocks, the headings of 
information maps are called “map titles’’ and the labels of information blocks are called “block labels’’. The purpose of 
both types of labels is to help describe the specific content of the subject matter, the functions of the block or the 
map, and combinations of these two types (Horn, 1993). The importance of labeling enables the readers to read the 
content of the subject matter quickly and understand the structure of the document; to find the important points that 
readers are looking for; to manage intermediate stages of information gathering and analysis in a mo re efficient way; 
and to retrieve information from the content efficiently. Further revisions of block labels may be necessary after the 
blocks are formulated into information map. This addition stage is to ensure the consistency in labeling between 
information blocks and the overall logic of the information map. 
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Consistent Formatting 

Consistent use of information blocks, block labels, and map titles in the formatting of structured-writing 
enhance readers’ ability in scanning documents efficiently. Readers will be able to locate the content easily, scan the 
important information and skip the details when time is limited. While the formats were initially developed to aid the 
reference and rapid scanning modes of use, these similar formats also aid learners to pre -organize and size up their 
learning tasks during the learning process (Horn, 1993). Consistent map titles provide essential information to orient 
the readers while block titles provide basic description of the information blocks. Readers immediately recognize how 
easy it is to scan and skip to what they want to read. They recognized the rapid access that the formats provide on 
every page. Therefore, they begin to have confidence in the document and in their ability to understand what it 
contains (Horn, 1993). 

Advantages of Structure- Writing Strategies 

There are a number of advantages in applying structured-writing strategies in developing textual 
information, which includes: easiness to scan, better information retrieval and recall, effective writing techniques and 
useful in designing just-in-time training. 

Easy to scan 

As the originator of structured-writing, Horn claims that structured-writing strategies are capable of first- 
pass sorting of 80 percent or more of the content of every subject matter that it has been applied to (Horn, 1993). 
Therefore, utilizing various stmctured-writing strategies can effectively organize, visually present (Whiteside & 
Whiteside, 1993) and simplify the complex information into usable models. Balan (1989) also agreed with Horn in that 
effective text design can grab the attention of readers and help readers to access, process and recall information. 
Balan created a number of text layout examples in her article to demonstrate to the readers the differences in using 
different cues such as listing; side-by-side text; justified vs. unjustified text; verbal cues such as summaries and 
analogies; and typographical cues such as boldface, italics, and underlining. Although scientific research was not 
performed, the examples have provided readers with clear illustrations of the importance in utilizing different cues in 
textual materials. Whiteside and Whiteside (1993) suggested that structured-writing aids the readers’ comprehension, 
retention, retrieval, and focus on the information. 

The development of structured-writing materials needs to go through a series of analyzing, chunking, 
labeling, and organizing versus the development writing conventional paragraph-written materials go through a 
simpler production process. Thus, it is not difficult to understand why structured-writing produces high-quality 
instructional materials, enhances the information processing skills and eventually improves learning and 
understanding of readers. 

Better information retrieval and learning 

Jonassen (1981) conducted a research study in comparing the abilities of students to recall (close-book) and 
retrieve (open-book) information between the programmed instructional materials and the structurally -written 
materials. The participants were 41 graduate and undergraduate students in an introductory media course. The 
results of the study suggested that both programmed instructions and structured-writing materials produced 
significant amounts of learning (recall) and there were no differences between the effectiveness. However, there was 
a very significant difference between the effectiveness of programmed instructions and structured-writing materials 
in the retrieval task. The participants in the structured-writing group performed significantly better than the 
programmed instruction group in the retrieval examination. The average test score for the structured-writing group in 
the retrieval examination is 33.9 and the average score for the programmed instructions group is 22.94, with a more 
than 10 points differences (t = 4.14, p<0.001). 

Additional studies have shown the effectiveness of structured-writing techniques in teaching. Fields (1982) 
conducted a study on a Master of Business Administration course about critical path method. It demonstrated that 
the participants regarded it favorably as a teaching method. The objective of the study is to analyze the responses of 
the participants and see whether significant learning took place. The study consisted of a pre-test and a post-test 
questionnaire, the percentage of participants that responded correctly increase from 29 percent in the pre-test to 78 
percent in the post-test, with a statistical significance at the 0.001 level. Therefore, the study concluded that there 
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was a significant gain in the average participants’ knowledge as a result of studying the structurally written text. 
Therefore, Fields concluded that besides that being a significant overall increase in the proportion of correct answers 
in the post-test, the level of knowledge of the participants had also been brought to a more uniform level. 

Effective writing techniques 

Structured-writing assists writers to develop various textual materials more efficiently as the techniques help 
to significantly decrease the length of any textual materials. Studies have been done to conclude that forty 
information blocks can be used to sort 80 percent of the sentences found in writing about most relatively stable 
subject matters such as sentences found in textbooks. This ability to provide such precise functional descriptions 
has been used in the design of various training and reference documents (Horn, 1999). 

Just-in-Time training 

Before the introduction of structured-writing techniques, few practitioners distinguished clearly between the 
initial instructional materials for learners during training and the reference materials that learners need in their 
performance context (Horn, 1993). However, it is increasingly difficult for readers to comprehend all the information 
around them. This effect is inevitable as readers are bombarded by excessive information, which changes faster than 
they could learn it. Therefore, just-in-time trainings emerge and gain importance because they [referant? training?] 
could provide quick revisions to learners after the initial training and before performing the skills in real-life. 

Structured-writing techniques can be applied to analyze and organize current trainings and transform them 
into just-in-time training. The techniques enable instructional designers to analyze and present all the information of 
the initial training precisely in a short and easy-to-read format for learners to use as reference. Information may 
include but not limited to: the prerequisite skills needed, objectives of the training, procedures that learners need to 
remember and pay attention to, safety notes, and even evaluation criteria so the learners can perform self- 
assessments. 

Other advantages of structured-writing techniques 

Improvements in reading speed — in a user survey administered to the US Air Force Reserve asked if 
structured-writing helped to improve their reading speed, 85 percent of the respondents agreed and perceived that 
their reading time decreased when reading structurally written materials. Another study was conducted to compare 
the US Army Officers in reading stmcturally written materials versus conventional, paragraph-written text materials; 
there was a 12 to 21 percent of improvement in reading speed when they were reading the structurally written 
materials (Information Mapping, 1999). 

Reduction in time in preparing materials — as writers apply structured-writing techniques in developing and 
writing textual materials, the time needed is reduced because the number of words used to write by applying 
structured-writing techniques is less than the conventional method. In a study of two separate US Army training and 
maintenance manuals, the word count was decreased by 39 and 53 percent for stmcturally written material when 
compared to conventionally written document on the same topic. 

Challenges of Structured- Writing 

After describing many advantages in utilizing structured-writing techniques in analyzing and organizing 
textual information, some researchers had pointed out some challenges of creating and using structured-writing. In 
order to have a better understanding of the techniques, it is important to acknowledge these challenges as well. 

To authors and writers 

As information is changing faster than ever, authors are going to encounter a lot of challenges. Some of the 
general challenges include: the increasing complex information to would need to be analyzed and organized 
efficiently using the structured- writing techniques; the authors will be responsible to construct structurally written 
materials for an ever-widening audience and they need to ensure that all the readers can understand the complex 
information easily; and the authors will need to be able to apply the structured-writing strategies to various delivery 
media such as papers and computers (Information Mapping, 1998). 
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To users and readers 

Resistance to new techniques — Hartley (1982) suggested that it would be challenging for readers to accept 
structured-writing as the new strategy in disseminating information because readers must learn the new reading 
technique. He suggested that readers could be hesitant to change their reading method at the beginning. Moreover, 
he pointed out that people might be “suspicious of once-for-all solutions.’’ 

Lack of motivational strategies — In addition, structured-writing does not specifically address motivational 
issues specifically for readers . This may be due to the fact that when Horn developed the structured-writing 
techniques, the target audience is executives and employees in a business environment. Therefore, it can be safely 
assumed that they will be motivated to learn (Fields, 1982). However, when trying to apply the techniques to students 
or the academic environment, it cannot be assumed that the students will be motivated to learn at the beginning. The 
failure to deal with the motivational issues could become a potential weakness for applying structured-writing 
strategies in an academic environment. 

Not focus on disseminating detailed information — Fields (1982) also pointed out that structured-writing 
focused on the clarity in organizing textual information but pay little attention of the depth of the information. From 
the standpoint of business and industry, it is understandable that people try to communicate in short and clear 
messages. However, in academic settings, the lack of depth could be a serious defect. For most of the subjects, 
students should learn more than a general idea or concept and be able to explore details in order to master the skills 
and understand the knowledge. 

Applications of Structure- Writing Strategies in Edueation 

During the development of structured-writing materials, teachers have to conduct a throughout content 
analysis. It required a considerable effort in clarifying any ambiguities in identifying and filling any “missing 
information’’ between each main point. These gaps might not be as obvious in a conventional paragraph-written 
materials (Fields, 1982), but could be easily identified by in structured-writing because of the information block format 
with one key point in each block. The format forced the teachers to recognize the disconnections between key points. 
Therefore, teachers would be able to produce high quality educational materials, teach smoothly and avoid 
discontinuities in classes. In order to examine the force of this claim. Fields carried out a simple test to himself by 
trying to produce a structurally written text on the battle of Waterloo, which he had almost no subject matter 
expertise. Upon the completion of the test, he concluded that structured-writing constituted a significant aid to the 
teachers in clarifying their ideas (Fields, 1983). However, he mentioned that the only doubt is whether it always has 
this assisting effect and whether there are alternatives that might help teachers even more. 

Conclusion 

Structured-writing has been developed to analyze and organize complex information into easy-to-read 
information blocks and information maps. Horn (1999) explained that structured-writing provides a systematic way of 
analyzing any subject matter to be conveyed into a written document. Information blocks are small chunks of 
information that are similar in that cluster sentences have strong relationship with each other. Forty such information 
blocks can be used to sort 80 percent of the sentences found in writing about most relatively stable subject matters 
such as sentences found in textbooks (Horn, 1999). This ability to provide such precise functional descriptions has 
been used in the design of various training and reference documents (Horn, 1999). 

The four basic principles of information blocks are : chunking — group all information into small units; 
labeling — name the blocks according to the content and key concepts; relevance — keep one key point within each 
block; and consistency — use similar wordings, terminologies, formats, and organizations. As a result of these 
principles, information blocks become far more usable text to scan, to read, and to memorize (Horn, 1998). Balan (1989) 
suggested that by carefully structuring text and applying typographical cues, the learner spend less time processing 
text and more time learning new material. Hence, improves the instructional value of a training material. 

Advantages of structured-writing included but not limited to: easy-to-scan; better information retrieval and 
understanding; effective writing techniques; decrease in reading time; reduction of time needed to prepare textual 
materials; and assist in the development of just-in-time training. Some of the challenges included: increasing in 
complexity of information that needs to be presented; meeting the needs of a wider range of readers; resistance from 
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readers to adopt a new reading technique; that motivational issues are not addressed; and lack of attention in 
incorporating detailed information. 



Executive Summary of Stmctured-Writing 

Definition • Stmctured-writing is a technique in formulating, designing, and writing text to 

improve comprehension and retrieval of information. 

• Even though the world of business and technology is growing so complex and 
information is changing faster than we could learn it provides a systematic way to 
analyze, organize and design information in written documents. 



Principles 



• Information Blocks — the smallest unit in structured-writing and each block consist 
of one key concept. 

• Information Map — a collection of one to nine information blocks with related 
concepts that group together in disseminating information clearly to the readers. 

• Labeling — each information block and information map is labeled to describe the 
main points. 

• Consistent formatting — the wording, formats, and labeling in the information blocks 
and maps should be consistent throughout the text to assist readers to locate the 
information efficiently. 



Advantages 



• Easy to scan — Horn claims that structured-writing strategies are capable of first- 
pass sorting of 80 percent or more of the content of every subject matter that it has 
been applied to (Horn, 1993). 

• Better information retrieving and understanding — students’ performance improved 
significantly using structurally written training materials (Jonassen, 1981). 

• Help to develop just-in-training — structured-writing enables instructional designers 
to analyze and present all the information of the initial training precisely in a short and 
easy-to-read format for learners to use as reference. 

• More effective writing techniques — forty information blocks can be used to sort 80 
percent of the sentences found in writing about most relatively stable subject matters 
such as sentences found in textbooks. 

• Decrease in reading time and level — participants perceived that they need less time 
to finish reading the same material written with structured-writing techniques. 



Challenges 



• Increasing amount of complex information that needs to be presented Meeting the 
needs of a wider range of readers 

• Lack of motivational techniques in structured-writing 

• Lack of attention in incorporating detailed information 
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Abstract 

Currently much energy is being used to create the technological framework necessary for an object-based 
approach to learning system development. However, there appears to be little consideration of the changes in 
analysis and design thinking required for the move towards object-based systems. This article provides a view on 
what changes are required and establishes a relationship between learning objects and performance support. The 
vision for learning objects is that rather than build every new system from scratch, systems will largely be 
constructed of reusable objects derived from object repositories on the web. Efficient construction and 
identification of useful objects requires, at minimum, knowledge of learning design strategies and principles. More 
importantly it requires identification of performance problems and related performance support tools that will 
address those problems at both organization and individual levels. To link this problem-oriented approach to the 
identification of proper human performance solutions, a holistic planning approach is recommended. 



A Framework for Object-Oriented Collaborative Analysis 

The aim of this paper is to describe processes that are required to support design, development, and 
evaluation of performance improvement solutions and the features of a methodological framework supporting the 
specification of reusable performance improvement materials. In recent years, the academic and workforce 
development communities in the United States have become increasingly concerned with the cost effectiveness of 
student and employee development processes. Efforts to improve the return -on-investment (ROI) of such 
development have been hindered by a craft orientation to the design and construction of learning and performance 
support systems (Clark & Estes, 1999). Such an orientation is expensive since it relies on the knowledge and 
experience of a few expert performers rather than using a standardized approach that is accessible to all performers. 
One solution to this problem has been to enhance the reuse of learning and performance support materials that had 
been previously developed. The idea behind reuse is that new systems will largely be constructed of standardized, 
reusable objects derived from object repositories on the Web rather than built from scratch in a craft -oriented manner. 

Studies of ISD professionals in a variety of working environments suggest that time pressures and 
management impatience with the ISD process result in an ISD practice that looks a lot different than the theory 
(Rossett & Czech, 1996). Likewise, the preparation of instructional designers may fundamentally limit the ability of 
designers to effectively impact or add value to the organization by focusing exclusively on learning. Davies (1989) 
found that designers in the private sector were generally thought to be competent but lacking in promotability due to 
their lack of business orientation and inability to take proactive approach to solving organizational performance 
problems. 

In addition to the limited impact of ISD practice on the workplace, there is a growing discontent among many 
practitioners with the ISD methodology. In a recent article (Gordon & Zemke, 2000), ISD was criticized for being too 
slow, not necessary, leading to bad solutions and having an outdated world-view. Whatever the merits of such 
claims, the underlying premise of ISD that a systematic approach to design should be used remains valid, therefore, if 
you reject traditional ISD you should have something to put in its place. 
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Emerging Trends 

Within the software -engineering domain, similar criticism of traditional design methodologies has resulted in 
a number of alternatives collectively described as ‘agile’ methodologies (Fowler, 2000). The agile approaches can be 
characterized as being a collection of suggested features, individual qualities and techniques, rather than a rigid 
definition of steps and outcomes. This is seen in the agile manifesto (http://agilealliance.org/), which is a collective 
statement of the common characteristics of agile methods. The manifesto includes statements such as: “We welcome 
changing requirements, even late in development. Agile processes harness change for the customer's competitive 
advantage’’. This contrasts with the approach in traditional methodology where there is a focus on defining and 
fixing requirements prior to building the software. 

Training, especially classroom training, has traditionally been the dominant approach to developing human 
performance in organizations. Pioneers in the field of performance technology such as Thomas Gilbert, Robert Mager, 
and Joe Harless have long advocated the view that organizational system elements such as expectations and 
feedback, tools and equipment, rewards and incentives, and motivation must be analyzed to identify barriers to 
performance. The essence of this approach is to identify valuable behaviors that produce measureable results and to 
remove both individual and work environment barriers to performance (Gilbert, 1998). A study conducted with over 
1000 organizations by Huselid (1995) found that human resources practices such as employee selection and 
recruitment, performance management, incentive systems, employee involvement, and training combined to 
significantly impact turnover, productivity, and short and long term financial success. This study is significant in that 
training was but one of many strategies impacting organizational results. By extension, efforts to develop 
technologies to create and use reusable objects to develop the performance of military personnel should look beyond 
the limited paradigms of traditional training and personnel development. 

More recently, a performance-centered approach to design of electronic performance support systems 
(EPSS) has emerged (Raybould, 2000; Dickelman, 2000; Gery, 1991). This approach relies quite heavily on analysis of 
the performer and the total performance environment in an attempt to design solutions intimately tied to the work 
context. Potential barriers to performance are removed as part of this design and learning is considered a 
consequence as well as a precursor to performance.. Thus, performance support design is somewhat of a blend of 
instructional systems design, performance engineering, and software design. 

Efficient identification, constmction, and reuse of information objects requires, at minimum, knowledge of 
learning and performance design strategies and principles. While much attention is being given to developing the 
methodological framework for an object-based approach to learning system development, there appears to be little 
consideration given to the changes in analysis and design thinking required for the move towards object-based 
systems. Such systems require analysis of human performance problems and identification of related performance 
support tools that will address those problems across organization and individual system levels. Lessons from 
software development have shown that it is not enough to have object-oriented construction technologies and 
standards; reuse and object thinking must also permeate analysis and design thinking. 

Framework 

In this paper, a methodological framework for designing performance support as an alternative to the 
traditional ISD model is described. Traditional ISD thinking primarily focuses on two human performance support 
solutions, i.e. education and training. There are a number of emerging performance support alternatives to training, 
e.g. knowledge management systems, personal digital assistants or wearable computer to access on-the-job 
performance support, automation of tasks and redesign of equipment with improved human factors engineering. 
Traditional approaches to training that assumes the learner is a passive recipient of knowledge are no longer 
effective. Eor many of the new roles within an organization, information presented in classrooms will quickly decay; 
therefore problem solving skills and self-directed knowledge acquisition are more valued. Traditional methodologies 
also tend to be oriented towards developing integrated stand-alone systems rather than systems constructed from 
pre-existing reusable components. There are now a number of technologies and emerging standards that enable a 
reusable object approach to the construction of performance support systems (e.g. www.adlnet.org) . 
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We deliberately use the term framework rather than methodology. A framework is meant to provide a 
structure for a variety of methodologies tailored to specific groups or situations, rather than a set of rules for a single 
correct way of developing systems. 

The key characteristics of an object-oriented performance support framework include: 

1. A human performance-orientation rather than a solution (training) orientation 

2. Object-orientation to facilitate reuse through the emerging knowledge/learning object technologies 

3. A set of recommended features including: 

• System modeling 

• Collaborative development and peer review 

• Integration with repositories of reusable objects 

• Blended performance solutions 

• Parallel iterative development of solutions 

• Managed deployment with feedback 

• Performance-based improvement 

• Rationale management 

• Automated support tools 

Figure 1 illustrates a life-cycle model for the planning, construction and delivery of performance support 
systems. It envisions the development of support tools that facilitate collaboration and modeling for the planning 
stage of the framework. It is intended to complement current initiatives towards developing technologies and 
standards for reusable object-based design and construction. In terms of delivery, the document envisions an 
evolution of learning management systems towards performance support management systems based on the 
foundation of performance analysis. Newly developed learning and performance improvement solution packages are 
accessed, used, and evaluated in a continuous improvement feedback process. Thus, information, knowledge, 
learning and performance support subsystems are interrelated and accessible based on individual performer 
requirements. 







Figure 1. Life-cycle model for object-oriented performance support systems 
System modeling 

Prior to the development of a new system, it helps to create a graphical model of what you are going to 
build. In architectural design, for example, an architect will create a drawing of a building design. This drawing may be 
altered many times at relatively low cost before a final plan is used for construction. Once the building is constructed, 
alterations become difficult and costly. A similar approach is used in many design domains and is facilitated by a 
commonly used modeling notation. In learning and performance systems, modeling is not yet an established part of 
most methodologies and there is no standard modeling notation. In software systems development the Unified 
Modeling Language (UML) has recently become popular for modeling object-oriented software systems prior to their 
construction (Booch, 1999). Parts of this language, in particular use case modeling, are adaptable to the needs of 
object-oriented performance systems. 

Collaborative development 

Traditional analysis and design can be seen as an expert -centered activity, i.e. an ‘expert’ in design will work 
with a subject matter expert in a single geographical location to design a training solution. The Internet has the 
potential to greatly widen the scope for collaboration of others in the analysis and design process when combined 
with the modeling approach noted above. If models of problem domains and intended solutions are published openly 
using a standard, easily understood graphical notation, these can be inspected by geographically distributed 
stakeholders. Thus, a group of analysis and design experts, subject matter experts and end-users/performers can 
discuss and pass comment on models and identify flaws prior to final construction. 

Integration with a repository of reusable objects 

There are efforts underway to create repositories of reusable objects. The thinking of analysts and 
designers must be consistent with this object repository approach for reuse to be successful. Developers should 
begin thinking in terms of objects at the analysis phase rather than at the construction phase. We propose that 
analysis models should be framed in terms of analysis objects. Analysis objects are descriptions of requirements that 
can be matched against existing objects in repositories or form the specification for the construction of new objects. 

Blended performance solutions 

Currently, performance problems are mainly solved with education and training solutions. By identifying a 
number of potential solutions to performance problems it is possible, and appropriate, to create a blend of several 
solutions to a performance problem. Performance models created during analysis have many potential uses. They 
may be excerpted to create job descriptions and new employee orientation checklists. Developed solutions can also 
be re-purposed to develop collateral materials that will assist the performer when transferring or applying solutions in 
the workplace. For example, core skills might be taught in a training course, and a variety of reference materials 
created or excerpted from them. These job aids or knowledge bases provide performers with on-the-job access to 
knowledge and procedures for seldom occurring situations. 

Parallel iterative development 

Traditionally, analysis, design and construction of the whole solution system is completed before delivery. 
However, in an iterative development approach, different solutions can be selected and developed at different rates. 
An object-oriented systems approach allows for the decomposition of solution systems into sub-systems and 
objects, where parts of a system can be created largely from pre-existing objects. Thus some solutions may be 
identified and designed concurrently with ongoing analysis. This allows multiple, blended solutions to progress to 
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delivery at different rates from others which reduces the overall systems development cycle time, and demonstrates 
the added value of the analysis process. 

Managed deployment with perormance-based evaluation 

The performance support developer’s responsibility does not end with product delivery. A system of 
quality assurance should be in place, such that use of a system is managed and monitored with the resulting 
outcomes fedback into subsequent development and new versions of systems. Evaluation of the products that 
performers access in terms of their content quality and the value they add to the individual performer and 
organization, requires management oversight and participation. The test of an effective system is not in functional 
performance or user satisfaction, but in the closing of the performance gaps identified in the analysis across multiple 
levels of the performance system. There should be an iterative relationship between performance monitoring and 
performance support system analysis and design. 

Rational management 

Most performance problems have a range of potential solutions; despite this, developers often consider 
only one type of solution. It is not always the case that those making the decisions are unaware of the alternatives. It 
is more probable that political, historical and cultural factors dictate that there are certain solutions for certain 
problems. Thus, it is important for decision-makers to have a defensible approach that promotes out-of-the box 
thinking with careful consideration given to potentially cheaper and better alternative solutions that are also 
politically palatable. 

Rationale management (Moran and Carroll, 1996, Burge and Brown, 2000) is concerned with capturing the 
knowledge that leads to the construction of a system. In particular it requires an auditing of the decision making 
when choosing problem solutions. Most performance analysts will be able to produce reasons for their analysis 
decisions; the distinction in giving a rationale is that there should be the following: 

• A description of the issues that were addressed prior to the design decision. 

• A list of the alternative solutions considered. 

• The criteria used in the selection. 

• The argumentation used to support each alternative. 

• The decision. 

The benefit of rationale management is that it requires the performance analyst to use a rational process to 
make their decision-making explicit. It discourages decisions based on whim or prejudice; as such decision making 
will not have a clear rationale. 

Automated analysis and design support tools 

There are a now a large number of software tools that assist in the implementation of automated instruction 
and a number that manage the delivery of instruction via the web. There are relatively few tools relating to the 
systematic analysis, design and documentation that should precede construction and delivery, and none that 
incorporate the emerging object model. There is scope for creating the equivalent of computer aided software 
engineering (CASE) tools for object-based performance analysis and/or instructional design. 

Spector and Muraida (1997), Goodyear (1997) and Kasowitz (1997) review much of the previous work in 
automated design support tools. Spector and Muraida note that there is strong motivation to develop such tools 
given there is “a lack of instructional design expertise, pressures for increased productivity of designers, and the 
need to standardize products and ensure the effectiveness of products.” The tools referred to in these reviews are 
based on the traditional models of instructional design and were mainly experimental; there are very few commercial 
products of this type. In contrast to this situation, there is now a range of tools designed to assist with object- 
oriented software analysis and design and some of the thinking embedded in such tools could be used to create new 
tools to support object-oriented performance support systems. Although the ideal would be to have tools that 
support a range of the features referred to above there are some that can support individual features (e.g. 
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collaboration is supported by tools such as Microsoft’s Share Point Team services 
http://www.microsoft.com/frontpage/sharepoint/) 



Conclusion 

The object-oriented performance support framework incorporates an object orientation that matches 
analysis thinking with object technology. A great deal of work is underway to define conceptually and technically 
what learning objects are and how they work. This object approach to computer-based instruction follows in the path 
of object-oriented software development. The software industry has found that object thinking begins not when 
constructing objects, but much earlier in the process - with analysis (Due, 2002). For objects to be considered useful 
and reusable in a variety of performance solutions (both training and non-training), their characteristics must be 
considered at each step in the problem-solving process. This framework therefore suggests that organizations 
committed to reusable objects not rush to the construction of objects at the expense of object thinking in the analysis 
phase of problem solving. 

Many organizations have learned that merely providing more information or more training to improve 
performance is usually insufficient. When training is an inappropriate solution, the use of learning objects simply 
means doing the wrong thing more efficiently. Human performance technology (HPT) and object orientation (00) can 
affect each other in a recursive process: meaningful performance system analysis impacts how one creates and uses 
objects, and object orientation can facilitate performance analysis. Individually, each of these elements can provide 
benefits to an organization, but taken together they create a synergy, equipping the organization to respond more 
creatively, effectively, and efficiently to performance problems and opportunities. 

There is a need for greater consideration, research and collaboration on the issues incorporated within the 
methodological framework. The current framework is presented not as a definitive answer to the need for new 
methodologies, but as a starting point for debate and refinement of methodological thinking. 
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Abstract 

Teacher education programs often struggle with issues such as diversity and technology integration in field 
placements for future teachers. Distance education technologies offer capabilities that can be used to address these 
issues. Video conferencing allows pre-service teachers to observe and interact with K-12 classrooms at a distance, 
which can provide for virtual field experiences of diverse classroom settings and involving innovative uses of 
technology. New IP-based video conferencing systems, which support good quality video and audio over the 
Internet, offer greater flexibility at lower cost than older video conferencing technologies. Pilot projects as part of 
P3T3: Purdue Program for Preparing Tomorrow's Teachers to use Technology, a PT3 implementation project, have 
developed models of how to link pre -service teachers and classes with K-12 teachers and students using IP -based 
video conferencing. Advantages of this approach include: development of pre -service teachers' observational skills, 
linkages with diverse classroom settings, and opportunities for university students to practice developing and 
delivering instruction via this technology. Limitations include issues with school Internet firewalls, classroom audio 
and sometimes video problems, and lack of in-person contact. The technology offers promise for certain kinds of field 
experiences in teacher education. 

Introduction 

Teacher preparation programs are faced with a variety of challenges. Over the past two decades, a number 
of national reports have emphasized the need to improve teacher preparation (Carnegie Forum, 1986; Holmes Group, 
1986; Moursand & Bielfeldt, 1999; National Commission on Teaching and America's Future, 1996). Today, teacher 
preparation programs must prepare future teachers to meet national and state standards with regard to both content 
and pedagogy in an era when there is increased emphasis on performance. Teacher preparation programs must also 
help pre -service teachers learn to use technology and develop their understanding of diversity and multiculturalism 
(NCATE, 2001). Meeting these challenges may require new approaches. 

Field experiences have been identified as a key means to better prepare teachers for the diversity and 
complexity of today's classrooms (Goodlad, 1990). While field experiences are generally recognized as critically 
important, many colleges of education, particularly those in rural areas, have difficulty placing students in field 
settings that provide for needed experiences with, for example, diverse student populations and technology. 

Distance education technologies offer capabilities that can be used to provide needed experiences for pre -service 
teachers when appropriate field sites are not in close proximity. Purdue University has initiated an innovative project 
in the use of technology-enabled field experiences, part of a Preparing Tomorrow's Teachers to use Technology 
(PT3) implementation grant, to address key components of its teacher preparation program including understanding 
of classrooms and diversity, technology integration, and development of appropriate instruction for K-12 learners . 
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While distance education has traditionally been viewed primarily in terms of content delivery, i.e. a teacher 
at one site presents course materials to students at other sites, today's distance education technologies offer 
opportunities for other types of interactions between colleges of education and K-12 schools. For example, using 
distance education technologies, future teachers can observe and interact with K-12 classrooms from afar. This 
concept is not new; closed circuit television was used for observation of classrooms in teacher education programs 
as far back as the 1960s (e.g., Abel, 1960). In the 1980s, Iowa State University's Teachers on Television program 
showed that the observation skills of pre -service elementary teachers could be improved through training using 
microwave-based video connections to public school classrooms (Hoy & Merkley, 1989). However, these 
technologies were expensive and difficult to setup and maintain. Today's video conferencing technologies offer a 
flexible and cost-effective option for observation of and interaction with school-aged learners at remote school sites. 
This paperreports on a project, part of P3T3: Purdue Program for Preparing Tomorrow's Teachers to use Technology, 
which is exploring the use of video conferencing distance education technologies for enhancing teacher education 
by providing opportunities for pre-service teachers and classes to link with K-12 students, teachers, and classrooms. 

Background 

After five years of reform planning by its faculty and administration, the School of Education at Purdue 
University recently completed implementation of the final phase of completely restructured elementary and 
secondary teacher education programs. The new programs, which were launched with students entering teacher 
preparation programs in the fall of 1999, feature a cohesive set of block courses and practical experiences that are 
anchored by four strands - technology, portfolio assessment, diversity, and field experience. 

In Purdue's new teacher preparation programs, the technology strand is manifest in a required educational 
technology course followed by integration of technology experiences throughout other courses. To address the 
portfolio strand, each pre -service teacher develops a portfolio to document professional growth, for self-reflection on 
learning, and to provide the foundation for performance-based licensure. A large-scale, web-based, electronic 
portfolio system has been developed as another aspect of the P3T3 project. The diversity strand is supported by 
appropriate course work and by exposing pre -service teachers to various forms of diversity (e.g., socioeconomic, 
mral/urban, religious, cultural, special needs/gifted populations) during field experiences. The field experiences 
strand is supported by a Theory Into Practice (TIP) component that accompanies each block of courses in the new 
program. The TIPs provide our pre -service teachers with more and richer field experiences than were available in the 
past. The P3T3 project is helping to support implementation of the new programs and ensure that technology is 
integrated throughout the teacher preparation programs as originally intended. 

The overall goals of the P3T3 project are to (1) prepare pre-service teachers to demonstrate fundamental 
technology competencies, using technology as a tool for teaching/learning, personal productivity, communication, 
and reflection on their teaching, and (2) prepare teacher education faculty to teach pre -service teachers in 
technology-rich environments, modeling approaches that future teachers should use themselves. The project is 
meeting its goals via three complementary components: (a) faculty development and integration of technology in 
teacher preparation courses, (b) development of a electronic portfolio system to support production of portfolios by 
all pre -service teachers, and (c) creation of rich and diverse field experiences enabled by the use of technology. This 
paper focuses on the latter, specifically the use of distance education technologies to create virtual field experiences 
for prospective teachers. 

As one way to address the challenges of providing diverse and rich field experiences in a limited geographic 
location, Purdue's P3T3 initiative is making use of two-way video conferencing to link pre -service teachers and 
teacher education classes with diverse K-12 students, teachers, and classrooms. The purpose of this initiative is to 
explore various models for enhancing teacher preparation through these linkages between the university and 
participating K-12 schools . 

Two video conferencing systems are used in the project. Some of our partner K-12 schools are linked to an 
intrastate fiber optic video network called Vision Athena (http://www.visionathena.org). The Vision Athena network 
is managed by the Center for Interactive Learning and Collaboration, a partner in the P3T3 project. While Purdue 
University does not have a direct connection to the Vision Athena network, we can connect with schools on the 
network through a video bridge. The other technology we are using is IP-based video conferencing equipment from 
Polycom (http://www.polycom.com), which supports good quality video and audio over the Internet. This 
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technology is relatively affordable and very flexible, because a standard H.323 connection can be established 
between any two locations with access to a reasonably fast Internet connection (128 Kbps or better). Because of 
these advantages, it is emerging as the technology of choice in our project. 

We use two types of Polycom video conferencing equipment in our project. Room-to-room video 
conferencing is supported by Viewstation SP (point-to-point) orFX (multipoint) units. These compact units have a 
camera with panning and zooming capability that can be attached to any available video monitor and plugged into an 
ethernet jack for Internet connectivity. Prices start at about $2500 for the Viewstation SP. For person-to-person or 
small group to small group connectivity, we use the Polycom ViaVideo computer-based desktop video conferencing 
unit, which operates in conjunction with a Windows PC. While the ViaVideo camera is of lesser quality and lacks the 
panning and zooming capability of the larger Viewstation units, the inexpensive (about $400) ViaVideo unit adds the 
exciting capability for application sharing during video conferencing. Two pilot projects involving these IP-based 
video conferencing technologies are described in this paper. 

Class to Class Video Interactions for Beginning Pre- Service Teachers 

In the first block of the teacher preparation programs at Purdue University, teacher education students take 
two classes: Exploring Teaching as a Career and Multicultural Education. The classes are designed to provide a 
foundation for future education courses by helping the pre -service teachers to begin to understand the nature of 
teaching and schooling. The two courses share a TIP component, an early field experience, in which students 
typically travel to nearby schools to observe classrooms for a couple of hours each week. The goal of this initial TIP 
experience is to help students become familiar with the work of teachers, observe teaching, reflect on student 
diversity and its expression in the school setting, and begin to interact with learners. 

Unfortunately, because of Purdue's geographic location away from any major urban centers, opportunities 
for encountering diversity during field experiences are limited for pre -service teachers in our program. In addition, 
pre -service teachers often feel there is no need to understand diverse populations of students because they plan to 
teach in predominantly white and rural areas after graduation (Yao, 1999). The demographics of the areas they plan 
to teach in, however, are rapidly changing and diversifying (Glazer, 1997). As local placements reflecting ethnic, 
linguistic, and socio-economic diversity are difficult to find within driving distance, and as it is essential that 
students have these experiences, one of our pilot projects has endeavored to use technology to reach out to diverse 
classrooms in other parts of the state. 

The pilot project described here was implemented in order to link learning about technology with learning 
about multiculturalism by placing beginning pre-service teachers in diverse classrooms for their field experiences 
through the use of video conferencing and the Internet. In this pilot project, elementary pre-service teachers enrolled 
in a section of the first block's TIP participated in their field experience via video conferencing with a partner school 
in a diverse area of Indiana. Using Polycom Viewstation equipment, a faculty member (one of the authors) and 
students at Purdue linked with a teacher and students in a grade three bilingual class in a diverse inner city school in 
East Chicago. The two sites connected once a week throughout the semester for between one and two hours. 

During that time pre -service teachers observed the classroom, interacted with the children and teacher, and prepared 
a variety of enrichment activities which they taught using the interactive capabilities of the Polycom technology and 
the Internet. Prior to each interaction the teacher posted information about her current curriculum and suggestions 
for activities on her website. The pre-service students prepared mini-lessons based on the teacher’s guidelines in 
consultation with the faculty member who acted as a liaison for all parties. 

At the beginning of each semester, the pre-service teachers became acquainted with the technology. They 
learned to connect to the remote site, operate the remote controls for the cameras both at the Purdue site and the 
school site, and develop mechanisms to facilitate communication. The pre-service teachers practiced with the 
equipment by splitting into two groups and video conferencing between two local classrooms . The university 
students received a checklist of procedures to follow prior to connecting and while connecting for reference. 

Prior to beginning the actual video conferencing field experiences, the university class made a site visit to 
the participating school. During this trip, the pre -service teachers spent the day touring the school; meeting staff, 
teachers and students; and interacting with the students in the grade three class involved in the project. During the 
visit, the pre -service teachers used guiding questions in order to do a mini-ethnographic project aimed at uncovering 
their assumptions of the school and students. This visit allowed the pre -service teachers to gain first-hand 
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knowledge of the school and the students, which we believe helped to overcome some of the impersonal nature of 
remote connection. 

Following the site visit, the virtual field experiences began and continued weekly through the remainder of 
the semester. (This pilot project has been repeated during five consecutive semesters.) Initially, pre-service teachers 
spent time observing the classroom and getting oriented to classroom activities. During one semester, the first 
session was spent on introductions. Students in the grade three class and pre -service teachers brought in baby 
pictures and made riddles about “Who am I?’’ Pre -service teachers prepared Powerpoint presentations of their 
riddles to share. Some riddles were done in English and some in Spanish for the bilingual classroom 

A typical interactive session began with the classroom teacher teaching a lesson. Pre-service teachers then 
took turns, individually or in small groups, directly teaching mini-lessons to the students. These activities reinforced 
what the teacher was teaching in the class or enriched the curriculum. Over the life of this pilot project, pre-service 
teachers have had the opportunity to engage in a variety of activities, including teaching lessons on equal and 
unequal fractions using everyday objects, colorful graphs, and diagrams and charts; reading stories and providing 
follow up questions; researching information about Benjamin Franklin and presenting it to students in the form of a 
skit; and communicating with the students in both English and Spanish. One session was devoted to discussing the 
World Trade Center disaster. Purdue pre -service teachers and the grade three students wrote memoirs about where 
they were on 9/1 1 as part of a process writing activity. The teacher provided links on her website that the pre -service 
teachers used to see how to discuss sensitive topics with young students. 

Informal observations of the project to date suggest that it has had benefits. In each semester, the pre- 
service teachers quickly adjusted to the technology and learned how to manipulate the equipment. They readily 
followed the guidelines of the classroom teacher and the faculty member; however, most groups were not 
adventurous in exploring the capabilities of the technology to work with the students in the classroom. Journal 
entries indicated that they began to see technology as a tool that could be used for teaching and their own and 
others’ learning, personal productivity, and communication. Significantly, one benefit seems to be the development 
of pre-service teachers' classroom observation skills. These beginning teacher education majors come into the 
course as unskilled observers, but through the guidance of a faculty member who observes alongside them via the 
video conferencing, they become better observers themselves. In addition, the shared observational experience 
leads to opportunities for richer class discussions. 

The pre-service teachers understanding of diversity issues and how to teach children unlike themselves 
have apparently also grown as a result of the project. Diversity occurs is in every interaction, although often subtly 
and in ways students may not expect. Many pre-service teachers thought the classroom teacher would teach 
“differently” because she had diverse students. They also expected badly behaved students and indicated surprise 
at the well-maintained school and the well-behaved students. In interactions with the teacher and students and in 
interviews with the teacher, they learned that the teacher has high expectations, does not water down the curriculum, 
teaches in an integrated style, and builds on the students’ prior experiences. They have also learned a great deal 
about teaching, including how to teach in a bilingual setting and how to teach at a distance. 

On an end-of-semester survey during one semester, 15 pre-service teachers rated the overall video 
conferencing experience good, 4 were undecided, and 1 rated it poor. No one rated it either very good or very poor. 
The pre -service teachers saw advantages in the ability to connect to a diverse classroom site and learn about 
technology and distance education. Limitations included the lack of person-to-person interactions and technical 
issues such as connectivity difficulties, audio and video problems, and the time delay in verbal exchanges caused by 
the distance. 

The teacher in the participating classroom indicated some reservations about feeling as though she was on 
display and that the students “went a little wild” after the Purdue class signed off. On the positive side she felt her 
students benefited from contact with Purdue students, enjoyed the activities, and readily spent time preparing for the 
activities. She felt that involvement in the project has kept her own thinking fresh, and she is using her participation 
in this project as part of what she will compile for National Board certification. Her students have indicated a range of 
feelings, most of them positive. While some of the K-12 students have reported feeling nervous, they like the 
attention and feel like the Purdue students are friends. Overall, these preliminary results are encouraging. 

Small Group Instmction via Personal Video Confereneing 
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A second pilot project involving remote field experiences was developed as a part of a course, Production of 
Instructional Materials, open to both undergraduate and graduate students. A faculty member (one of the authors) 
launched this pilot project in cooperation with the P3T3 project to provide the university students with a genuine 
context and audience for the design, development, and implementation of instructional materials. Teams of university 
students created instructional materials for a K-12 audience. By linking with a K-12 classroom, an authentic context 
was created for university students needing to develop instructional design skills, and the K-12 students benefited 
by receiving custom-designed instruction. In addition, an apprenticeship model was employed as part of the project 
whereby advanced graduate students mentored the teams of university students, acting as project managers, during 
the instructional design process. 

Three different experiences were implemented during different semesters. In each case, the key project goals 
were to: (a) develop and implement web-based learning materials for K-12 learners, (b) develop and implement video 
conferencing distance learning activities with students at a remote site; and (c) through these activities, provide 
university students with the opportunity to enhance their skills in the design, development, and implementation of 
instructional materials. Video conferencing was used to a lesser extent than in the first pilot project, described above, 
but it played a key role in the project. Various forms of evaluation, including observations, student reflections, peer 
assessments, and instructor assessments were employed during the projects. 

In the first semester of this pilot project, a small group of university students created a web-based virtual 
field trip for a class of second grade students at a rural partner school who were planning to visit a children’s 
museum in the state. The virtual field trip was specifically designed as an online learning activity with both 
asynchronous and synchronous components. The main aims of the virtual field trip were to expose students to 
information prior to the actual field trip to the museum, help students get excited about going to the museum, increase 
student participation at the remote site by helping them obtain information about the predetermined trip, and set the 
educational tone and expectations prior to the museum visit. To accomplish these goals the virtual field trip was 
designed to help students obtain information online, assimilate the information, and communicate their 
understanding to peers, university students, and teachers. 

Video conferencing was used for both planning and part of the implementation of the lesson built around 
the virtual field trip. The university instructor and team of student developers used video conferencing on two 
different occasions to connect to the participating classroom teacher to collaboratively plan the virtual field trip and 
associated instructional activities . Then, video conferencing was employed for the university students to observe as 
the teacher implemented initial instructional activities with the second graders in the classroom. As a culminating 
activity, the team of university students used the video conferencing to remotely lead a final classroom activity 
involving the web-based instructional materials . Overall, the development and implementation activities spanned 
about ten weeks of class time, although only two hour-long video conferencing sessions were involved in the 
implementation phase of the project. 

The next semester, university students engaged in small group to small group interaction with a group of 
fifth grade students. During this semester, the team of university students developed a stock market investment 
project based on specific programmatic themes outlined by the cooperating in-service teacher. Adapting the 
approaches of the first semester, four lessons were created, one web-based and three via live video conferencing. A 
key strategy implemented during this second iteration of the pilot project was the use of application sharing during 
video conferencing using Polycom ViaVideo units. This capability allowed the university students to share an 
application (Microsoft Excel) remotely and work with the K-12 students to co-construct a spreadsheet as part of the 
lesson. The purpose of the application sharing was to promote a higher level of interaction between university and 
K-12 students during the lesson. 

The third semester of this pilot project continued the use of application sharing as one part of a 
comprehensive lesson developed for a fifth grade class at a partner elementary school in an urban center in the state. 
This time, the team of university students directed by an advanced graduate student developed a lesson involving 
both web-based instructional materials and synchronous video conferencing sessions built around a popular 
engineering contest founded at the university called the Rube Goldberg Machine Contest. Rube Goldberg was well- 
known cartoonist who drew cartoons of absurdly complex machines that took many complicated steps to perform a 
simple end result. In the early 1980s, students at Purdue University began the contest that paid homage to the 
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cartoonist by requiring teams of engineering students to construct elaborate machines that required many steps to 
perform a simple task such as sharpening a pencil or screwing in a light bulb. 

The university team developed a website to provide background information for the fifth graders about 
Rube Goldberg, metric measurement, and simple machine concepts. Video conferencing sessions were used to 
introduce students to the lesson concepts building toward the culminating activity of the lesson, a Rube Goldberg 
machine building contest for the fifth graders. Application sharing with the ViaVideo unit was used during one of the 
video conferencing sessions to help students understand simple machine concepts. Sharing a photo of a simple 
machine, such as a pair of scissors, students were able to manipulate images in a Microsoft Word document to 
demonstrate that they understood concepts of force, the location of the fulcrum of a lever, and so forth. 

During the final session of the lesson, teams of fifth grade students, who had constructed their own Rube 
Goldberg machines to raise a small American flag in multiple steps, demonstrated their machines in front of the video 
conferencing unit. The team of university students observed the activity at a distance and served as the judges of 
the contest. Thus, the university students were able to gain experience in the creation of authentic instruction from 
planning to design to development to implementation and finally evaluation. At the same time, the fifth grade 
students benefited from an engaging lesson that involved important mathematics and science concepts and 
interactions with a genuine audience consisting of college students. It was in many senses a win-win activity. 

These three iterations of this pilot project involved a number of important instructional attributes. The 
university students' learning was situated in real context that allowed them to develop important instructional design 
skills. The instructor and advanced graduate student modeled the instructional design process for the novice 
designers and coached them as needed. Under this guidance, the university students were able develop their own 
expertise, gradually assuming increasing responsibility for the instructional outcomes. The video conferencing 
played a key role in the process, both as a vehicle for planning with the cooperating teacher and as a delivery system 
for the lesson that resulted. It also stretched the university students by requiring them to plan and implement a 
lesson using what for most of them was an unfamiliar instructional environment. 

A formative evaluation that included interviews and surveys from the university students indicated that on 
the whole the experience was beneficial. They felt that the remote field experience was instructionally valuable, 
showed a real-life application, increased their technology confidence, generated satisfaction, better prepared them for 
teaching in the future, and engendered a desire to continue using technology for teaching. Challenges included the 
difficulties of jointly planning an instructional unit with a cooperating teacher at a distance, the constraints of a 
semester schedule, and students' lack of comfort with this approach and the ambiguities that resulted. There were 
occasional problems with the technology such as difficulties in establishing good connections and the limitations of 
ViaVideo unit as a video conferencing tool with small groups when it was designed mainly to be a person-to-person 
video conferencing tool. Overall, the remote field experience was judged a reasonable success, and both university 
students and the K-12 students enjoyed their involvement in the experience. 

Strengths and Limitations of the Distance Learning Technology 

When working with any new medium of communication, there are inevitable difficulties and a period of 
acclimatization. The first step in each of these pilot projects was simply to get the technology working on both ends. 
For IP -based video conferencing, a significant initial barrier is school Internet firewalls. Because of information 
security concerns, most schools are protected by an Internet firewall. While keeping people out of the internal 
network, a firewall can be configured to allow selected outside connections. When trying to set up this access, we 
ran into difficulties in both of these projects. These problems were resolved, but not without a fair amount of time 
and the help of several technicians. Once established, the IP-based connections worked fairly well most of the time. 

Once the equipment was working, the participants had to spend time learning to use the system to 
communicate. This process was not difficult, but it was critical to the success of each project. The university 
students needed to practice using the system to help them get a feel for the communication abilities, and the K-12 
teachers and students also needed to practice with the system to develop a level of comfort and fluency with the 
technology to facilitate routine communication. We found that it helped to develop conventions to facilitate 
communication, such as having name signs and using signals to denote when students at the remote site were 
supposed to do something. 
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In both of these pilot projects, pre-service teachers learned to see technology as a tool that enabled them to 
communicate across distance, and with students they may have had little experience of in the past. Exposure to the 
classrooms in East Chicago and our other partner sites seemed to open their minds to new possibilities. As a result, 
class discussions were rich and varied. In addition pre -service teachers learned to work together in groups and in 
partnership with a faculty member and classroom teacher. 

The main issue was that the pre -service teachers were not in a “real” classroom with “real” students. Some 
students , at least initially, felt a loss at this mode of interaction. Sustained interaction for a lengthy period, such as 
the two hours originally planned for the video conferencing with the diverse classroom in East Chicago, proved to be 
difficult for the pre -service teachers. So, that experience was restructured into shorter time blocks involving an initial 
discussion, observations, and then debriefing. In the other pilot project, students were somewhat uncomfortable 
with the open-ended nature of the process and the difficulty of communicating with a teacher at a remote site. 
Structure is needed to help pre -service education students benefit from this experience, but it is important to remain 
flexible and willing to adapt. 

The technology is good, but students in the pilot projects have noted limitations. IP-based video 
conferencing connections are susceptible to problems as a result of limited bandwidth or network congestion. 
Sometimes the Internet-based video conferencing connection is "choppy." When Internet packets are "dropped” as 
a result of network traffic, this can result in the video freezing and the audio breaking up. Even when working 
perfectly, the picture on the screen, while not bad, could be clearer. Effective observation often means noting subtle 
facial expressions and body gestures not easily discerned by viewing a video monitor. The ViaVideo unit, which is 
really designed for personal video conferencing, can be used with small groups, but the results are not optimal; 
without panning and zooming capability, the video is limited. 

Audio is as important as or more important than the video. While the teacher’ s voice comes through clearly 
most of the time, the children’s voices are less clear. Eurthermore, background room noise can create interference. 
While we have found that having the teacher work with the students to speak more loudly and clearly, audio quality 
is generally a problem that we have not fully resolved.. 

Considering all factors, as a part of a teacher education program, which has at its core emphases on early 
and continued field experiences, on developing technological skills , and on understanding diverse learners, these 
virtual field experiences seem to be a worthwhile way to expose pre -service teachers to experiences they might not 
otherwise get. Distance education technologies seem to offer significant promise for expanding the options for 
linking students in teacher preparation programs with K-12 teachers and students. 
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Abstract 

As instructional designers, we often fail to practice what we preach. Learn how the Academic Technology group 
at the University of New Hampshire applied the widely accepted ADDIE model ( Analyze, Design, Develop, 
Implement, Evaluate) to the instructional challenge of developing and implementing the campus’ first annual 
Eaculty Instructional Technology Summer Institute ( FITSI). 

Background 

The University of New Hampshire is a public land-, sea-, space-grant institution serving an undergraduate 
population of roughly 12,500 undergraduate and 2,500 graduate students. UNH offers 2,000 courses in more than 100 
majors and supports a broad array of undergraduate, professional, research, and graduate programs. The University 
pursues a three-fold mission of teaching, research, and public service. 

Computing and Information Services (CIS) at UNH provides infrastructure, training, and support 
for technology that sustains the academic mission of the University. Academic Technology, one of the five groups 
within CIS, fosters collaborative partnerships that advance the integration of technology in teaching and learning, 
and provides support and access to instructional technology resources for faculty, students, and researchers. The 
Academic Technology group addresses the needs of over 700 full-time and adjunct faculty, a constituency that 
spans the technology-skill spectrum— from highly technical early adopters, to faculty reluctant to embrace 
technology in teaching. 

In September 1999, UNH implemented Blackboard's course management system, giving many faculty on 
campus their first experience using technology as an integral part of traditional classroom teaching. Over the next 
two years, we sought ways to support this effort, and to develop and integrate other academic technology services. 
At the end of two years, over 50% of our full-time faculty were teaching more than 450 Web-enhanced courses to a 
student enrollment in excess of 22,000, but few had moved forward to embrace other instructional technologies. Our 
immediate challenge was to support Blackboard as a mission-critical application. Our ultimate goal, however, was to 
encourage faculty to incorporate a broad range of instructional technology tools into their teaching, not just build 
Blackboard courses. 

Our early experience with Blackboard convinced us that we needed a development model that would draw 
faculty into instructional technology theory and practice without a prohibitive investment of time. Analysis had 
shown us that: 

• The simplification of tasks accomplished through a course management interface clearly opens doors, 
raises awareness, and encourages preliminary exploration of the possibilities of instructional 
technology. It is not sufficient, however, to encourage faculty to explore other instructional 
technologies proactively. The sheer simplicity of the interface ironically preempts inquiry into other 
ways to use technology to enhance the traditional classroom. 

• For those who did look past Blackboard to other instructional technologies, the primary inhibitor was 
not skills or risk, but time — time to explore the theory, time to acquire the skills, and time to implement 
the practice. 

Thus we found that, although Blackboard was opening doors to the possibilities of instructional 
technology, in most cases those doors weren't leading anywhere. 
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This realization helped us define a new development model that would move beyond problem-oriented 
support and application-based training, to a team-oriented design-and-development approach that would draw 
faculty more deeply into instructional technology theory and practice without a prohibitive investment of time. Our 
new model would: 

• Encourage faculty to explore the broad possibilities of instructional technology; 

• Promote an awareness of instructional design theory and instructional technology tools; 

• Allow faculty to concentrate on pedagogical issues and content; 

• Provide skill-based design-and-development support; 

• Reduce the need for large investments of faculty time. 

In March 2001, in collaboration with the University Library, the Academic Technology group hosted a small, 
half-day symposium entitled “Meeting Instructional Challenges with Technology.” Open to faculty at large, the 
event was well attended and well received. The success of this modest effort suggested a faculty constituency 
interested in similar, but more substantive, events. Our initial research and discussions resulted in a proposal for a 
five-day, intensive Faculty Instructional Technology Summer Institute (FITSI) designed to address the needs of 
faculty who had taken their first tentative steps toward a creative and robust use of technology in teaching, and who 
were now ready to move to the next level. We sought and received funding from a number of sources, and the 
Institute took place during the first week of June 2002 on the UNH campus. 

The FITSI Model 

Our immediate objective in planning the Institute was to raise faculty awareness of instructional technology 
tools and relate the use of those tools to specific teaching and learning objectives. Our broader goal, however, was 
to foster a campus environment that would encourage but not coerce, engage but not intimidate, support but not 
direct faculty in the acquisition of instructional technology skills. Ideally, we wanted the Institute to become a 
regular feature of faculty development at UNH. 

Feedback from faculty who participated in “Meeting Instructional Challenges with Technology” indicated 
that, to be successful, a more extended Institute model would have to: 

• Provide incentives for faculty to explore the broad possibilities of instructional technology; 

• Promote an awareness and appreciation of the relevance and impact of instructional technology and 
instructional design theory; 

• Allow faculty to concentrate on pedagogical issues and content without having to invest an inordinate 
amount of time dealing with technology; 

• Motivate faculty who were successful implementing instructional technology to carry the message out 
to their colleagues; 

• Include an evaluation strategy to confirm the impact of instructional technology theory and practice on 
the teaching and learning process. 

We determined that our inaugural Institute would address the pedagogical principles that support the 
successful integration of technology into teaching and learning, rather than the details of the technologies 
themselves. 

We decided to ground the program in the widely acclaimed research of Chickering and Gamson detailing the 
“Seven Principles for Good Practice in Undergraduate Education” and Chickering and Erhmann’s subsequent work, 
"Implementing the Seven Principles: Technology as Lever," which further developed Chickering's theory by applying 
the seven principles to the field of instructional technology. In the process, we focused on the three most common 
barriers that prevent faculty from successfully integrating technology into their teaching: 

• Lack of familiarity with best practices, knowledge, and research on the appropriate uses of technology 
in the classroom; 

• Lack of technical skills, especially the design and production of digital materials for direct use in the 
classroom; 

• Lack of resources and time to analyze, design, develop, implement, and evaluate instructional 
technology projects. 

We began our planning process with a survey of the literature and an examination of viable models at other 
institutions. As our own model began to take shape, it became apparent that, if we were to achieve the goals we had 
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set forth, we would need a broad range of resources, the cooperation and support of a wide variety of people and 
groups on campus, and, above all, a clear plan to follow. We decided to apply the widely recognized ADDIE model 
of instructional design to our own instructional challenge. 

The ADDIE Model 

The ADDIE model supports a systematic approach to analyzing and meeting instructional needs. ADDIE, 
an acronym for Analyze, Design, Develop, Implement, and Evaluate, facilitates a series of steps that guide 
instructional designers in the planning and development of instructional projects. Each step defines activities, 
identifies relevant data, and answer questions crucial to the successful implementation of the project. Eor example, 
the analysis phase provides information about learner characteristics and environmental factors or limitations. 
Although commonly applied to instructional or training challenges, we relied on ADDIE as a useful anchor to help 
the Institute planning team define objectives and follow a shared set of principles and procedures as they worked 
toward a common goal. 

Analyze 

No amount of planning or effort can compensate for a lack of acknowledgment by faculty that technology 
has something to offer them, or a clear demonstration from faculty support groups that the benefits of technology are 
accessible. Hence, our own planning process began with an examination of best practices, a careful assessment of 
current services, an inventory of available resources, and a detailed analysis of our audience. 

We turned first to the difficult task of obtaining participation and buy-in from outside Academic 
Technology. Our experience disclosed the following imperatives: 

• Seek support from the principal administrator with responsibility for those developing and 
implementing the event. Since Academic Technology is a part of Computing and Information Services, 
we sought and gained sponsorship from the Asst. Vice President for Computing and Information 
Services. 

• Involve other campus organizations and groups that share a common mission. In our case, we reached 
out to the Library and the Teaching Excellence program. Both groups participated actively in the 
planning and implementation of the Institute. 

• Enlist help wherever you find it. At UNH, each college and school has at least one Academic 
Technology Liaison (ATL) who manages the information technology needs of the faculty and staff. 

We invited the ATLs to join in the planning and implementation of the Institute. 

• Promote interest among Deans and department Chairs. The endorsement of Deans and Chairs lent 
weight and credibility to our effort. This is especially important for first initiatives. It is also useful to 
enlist the support of CIOs, Provosts, and other senior administrators who can tie your efforts to the 
overall mission of the institution. 

• Beg, borrow, or steal all the funding you can, from whatever sources you can find. Most important - 
secure commitments for this funding early, so that your planning does not outstrip your resources. Our 
Institute was underwritten by a variety of sources: the Academic Technology group. Computing and 
Information Services, a campus fund called the Eaculty Instmctional Technology Development Grant, 
and even some Deans who strongly supported our effort. 

• Inform and involve mid-level managers and staff. Their contributions are crucial to the success of the 
initiative. Our AT professional staff researched topics and technologies, developed and delivered 
hands-on workshops, produced documentation and Institute materials, and generally assisted in all 
aspects of logistics and implementation for the Institute. This is time -intensive and requires attention 
to detail, but without these supporting efforts, all your planning may be for naught. 

Once we had commitments of support from as many quarters as possible, we were ready to determine how 
far that support could carry us. The next step was to analyze our audience. A careful scrutiny of our constituency 
revealed these significant facts: 

• Our faculty — despite prevailing wisdom to the contrary - are risk-takers. Our training population did 
not consist entirely of early adopters, but shaded rather heavily over into early- and even late-majority 
users. 
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• Contrary to our initial instinct, an elaborate training model can, in itself, be intimidating. Faculty tend to 
extrapolate from the intensity of the training to the difficulty of the process -- "If it really requires that 
much training, it's too complicated for me.” 

• Training is often perceived as an obstacle to participation rather than as an enabling experience. 
Lengthy training, in fact, discourages faculty and acts as a disincentive to explore new ideas. 

• Faculty identify the support they receive after training as the key to success. They are willing to 
experiment, stumble, innovate, and even fail, if they know there will be someone available to address 
their immediate concerns and ensure that their content is delivered intact. 

The combined results of our review of the literature, our survey of available resources, and our analysis of 
our constituency ultimately led us to define six clear design objectives. A successful Institute would: 

• Provide incentives for faculty to explore instructional technology; 

• Promote an awareness and appreciation of theory as well as practice; 

• Allow faculty to concentrate on pedagogy and content; 

• Follow-up on Institute topics and provide ongoing encouragement and support; 

• Motivate faculty to share their experiences with their colleagues; 

• Evaluate the effectiveness of the model and confirm its impact. 

Design 

Our research had shown us that the most successful faculty development models shared one trait - they 
subordinated technology to a clear emphasis on pedagogy. We thus self-consciously focused on the pedagogical 
principles that support the successful integration of technology into teaching and learning, rather than the details of 
the technologies themselves. We structured the Institute to combine nationally recognized keynote speakers, 
presentations by experts in the field of instructional technology, videoconferencing, and a variety of hands-on 
workshops targeted on specific instructional technology skills and specific pedagogical concerns. Participants were 
chosen through a rigorous application process to identify those candidates best equipped to carry the Institute's 
message back to their home departments and colleges. We initially planned to accept two faculty members from each 
of the five colleges at UNH. In the end, however, support from college Deans was so strong that they came forward 
with funds to support additional participants. 

A common adage among training professionals claims that “If you feed them, they will come.” While this is 
generally true, we felt that faculty would need to be “nourished” by more than a few free lunches to commit 
themselves to five eight-hour days of intensive, total-immersion training. The most successful models we examined 
in our design phase all offered useful and substantive incentives to participants. 

As incentives, we considered stipends, student support, PDAs, and laptop computers. In the end, we 
offered faculty the choice of a Dell or Macintosh laptop, fully equipped with Microsoft Office applications and a 
carrying case. The laptop was explicitly offered as the personal property of participants, to be used for professional 
or personal purposes, as they saw fit. Although clearly the laptop computer provided a strong incentive for most 
faculty who attended the Institute, it was gratifying to see that most focused on how they could use the laptop in 
their teaching and research. One or two even commented that they would use the laptop as a departmental resource, 
giving other colleagues access to it. This clearly supported our goal of disseminating information about technology 
and instructional design principles to the widest possible audience. 

In return for this support. Institute faculty agreed to participate in three follow-up activities during the 
upcoming academic year: 

• Work with Academic Technology and their college and departmental colleagues to promote the use of 
instructional technology; 

• Assist in conducting a college-wide needs assessment to provide quantitative data for Academic 
Technology and also serve as a planning instrument for the next year's Institute; 

• Communicate their experiences at three Faculty Instructional Technology Summer Institute meetings or 
events held during the subsequent academic year. 
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Develop 



With our design objectives firmly in hand, we began to organize our resources and target them on specific 
aspects of the Institute model. Our design clearly called for a collaborative effort. We convened a development 
committee that represented all sectors of our Academic Technology group and representatives of the Teaching 
Excellence program and the University Library. This gave us access to videographers, photographers, instructional 
designers, trainers, and systems and faculty support personnel, as well as a cmcial faculty and research perspective. 
This committee drew the outlines of a week-long program, considered incentives, addressed costs, identified 
resources, sought funding, and developed an agenda that included keynote speakers, videoconference 
presentations, and a series of hands-on workshops. A key focus of the development process was to address all six 
objectives identified in the design phase (incentives, awareness, pedagogy, follow-up, sharing, evaluation). 

We self-consciously set out to address the three primary inhibitors to technology integration identified in 
our analysis phase. 

• The first barrier, lack of familiarity with best practices, we addressed through presentations by two of the leading 
experts in the field. Both delivered four-hour presentations on the first day of the program to establish the 
foundation of theory and research upon which the Institute built. The presentations helped guide faculty as 
they began to build a "toolkit” of best practices (including the ADDIE model) and techniques to apply to 
instructional challenges throughout the week. 

• The second barrier, lack of technical skills, was addressed by a series of hands-on, interactive workshops that 
built on the materials presented the first day. These focused on putting the teaching/learning theories explored 
on the first day of the program in context, showed faculty how to integrate instructional technologies 
successfully into their curriculum, and encouraged discussion of how these technologies can enhance learning 
outcomes. 

• The third, and perhaps most resistant, barrier for faculty is the lack of time and resources. We addressed this 
barrier by showcasing the wide range of resources and services available to UNH faculty through the Academic 
Technology group. We demonstrated how these resources could be matched to specific instructional 
challenges, making it possible for faculty to think creatively about the use of technology in the classroom 
without a prohibitive investment of time. 

To maintain the emphasis on pedagogy, we bracketed our practical, hands-on workshops between 
presentations by nationally recognized experts in the theory and principles of active learning and the systematic 
design of instruction, regardless of whether that instruction was technology-based. Mid-week, we engaged two 
additional guest speakers, thus modeling tele - and videoconferencing, and Internet II technology. We concluded the 
week with a guest speaker who provided a retro - and pro-spective look at where instructional technology has been 
and where it may take us. 

Interspersed throughout the week were twelve hands-on workshops on topics that supported the principles 
and concepts discussed on the first day: 

• An Overview of On-line Communication Tools* 

• An Overview of Academic Technology Support Services* 

• Adapting Course Materials for a Digital Environment 

• The Visual Display of Information 

• Using Technology to Create Collaborative Workspaces 

• Using Internet Resources: Tool’s Gold or 24-karat? 

• Technologies that Support Active Learning* 

• Managing On-line Discussion* 

• Using Threaded Discussion to Encourage Critical Thinking 

• Opening New Channels for Eeedback Using Technology 

• Assigning and Evaluating a Video Project 

• Creating and Using Images in Your Classroom 

Lour of these were designated “core” sessions (*) that all participants attended; the remaining eight were 
developed as “breakout” sessions from which faculty chose. In addition, all participants engaged in a panel 
discussion led by faculty currently using technology in their teaching. 

Workshop developers were required to meet five criteria: 
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• Communicate clear objectives and identify which of the seven principles was addressed; 

• Provide a firm grounding in theory and research, including a bibliography of key sources; 

• Provide sufficient documentation so that faculty who did not attend the session could learn from it; 

• Provide hands-on activities that would allow faculty to experiment with the target technology; 

• Provide a PowerPoint presentation for the Institute Web site (http://at.unh.edu/fitsi/2002). 

Evaluations from the Institute indicate that the decision to subordinate technology to teaching was crucial 

to the success of the week. 

Our stated intention from the inception of our Institute plan was to establish an event that could act as a 
springboard for future faculty development programming and events. Following up with participants was thus an 
essential part of our overall plan. To ensure that the Institute would indeed have a long-term impact, we built five 
follow-up mechanisms into the initial design. 

• A “reunion” scheduled for later in the summer at which participants would help us lay the groundwork 
for future instructional technology efforts; 

• The “shadowing” of one or two participants by Video Services to track their ongoing development and 
document how they implemented the techniques learned at the Institute; 

• An event scheduled for late in the fall at which we would communicate our plans for a college-wide 
instructional technology needs assessment; 

• Involvement of all participants in this needs assessment, including distributing the instrument, 
communicating its objectives to their colleagues, and helping us to collect and analyze the data; 

• A spring event at which participants would share their experiences with colleagues and welcome 
participants for the next Institute. 

Another design objective was to ensure that Institute faculty helped spread the word, not only about their 
own experiences at the Institute, but about the impact that instructional technology has had on their teaching. To 
this end, we encouraged feedback, acted on suggestions, and solicited ideas for future programming. 

Implement 

The pool of expertise and talents available to us through Academic Technology professional staff was 
considerable. The key to successful implementation was to keep everyone’s eyes on the prize, to keep the Institute 
targeted on the six design objectives, and to keep the development and implementation process moving forward at 
the necessary pace. To coordinate the efforts of this diverse group, we established four committees: 

• Logistics: Identified and secured appropriate facilities and technical support for Institute events; 

• Presentations: Solicited abstracts for workshops and presentations, and ensured that Institute events 
addressed both the pedagogy and technology; 

• Materials: Designed and developed templates for all supporting materials and produced a 
comprehensive annotated bibliography for all workshops and sessions; 

• Advertising: Wrote press releases for on- and off-campus publications and developed the Institute 
Web site. 

These four committees worked together to move the planning and implementation process forward on 
schedule. A key directive was to coordinate all committee efforts toward the production of consistent, targeted, 
high-quality documentation for the Institute that would stand alone as a resource for faculty who did not attend. The 
net result was a high-profile professional presence that lent weight and credibility to all Institute events. 

In retrospect, we identified four key imperatives for the implementation phase: 

• Carefully cultivate an atmosphere of participation and ownership for all concerned; 

• Establish deadlines and adhere to them; 

• Clearly articulate all objectives; 

• Assign responsibilities for key tasks and hold people accountable. 

Evaluate 

Our desire to establish the Institute as a permanent part of the faculty development landscape at UNH made 
careful review and evaluation of our initial efforts crucial. Moreover, the successful models and programs we had 
examined all included a comprehensive evaluation plan as an integral part of their design. 
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We gathered altitudinal survey information on-site for each of the twelve hands-on workshops. The 
instrument consisted of twelve statements, accompanied by a Likerd scale, phrased to elicit to what degree faculty 
agreed with positive statements about the workshops. We also encouraged discursive comments from faculty, 
including what they liked best and least about the workshop. 

The statements were grouped under two main categories: 

■ Overall impressions of the workshop: 

o The workshop was well organized and easy to follow; 
o Objectives for the workshop were clearly communicated; 
o It was clear which of the seven principles the workshop addressed; 
o There was enough opportunity to interact with the presenter; 
o The workshop was about the right length; 

o The presenter was knowledgeable and presented the material clearly; 
o Overall, this was a good workshop. 

■ Workshop content, materials, and hand-outs: 

o The hand-outs were well organized and easy to understand; 
o The material presented met my expectations; 
o The information provided will be useful to me; 
o The hands-on exercises were clear and worthwhile; 
o There was a good mix of presentation and hands-on activities. 

To our surprise, faculty took the time to be thoughtful and reflective on these level-one evaluation sheets, 
and our response rate approached 90%. 

In addition to individual workshop evaluations, we administered an overall evaluation on the last day of the 
Institute that gave participants an opportunity to rate the effectiveness of each Institute event or session, including 
keynote and guest speaker presentations. We felt that, although this was somewhat redundant in the case of the 
workshops, it gave participants an opportunity to review their impressions in retrospect and from the distance of at 
least a day or two. We also gave them an opportunity to evaluate the logistics of the Institute, including venues, 
food, comfort, and overall atmosphere (see Table 1). 



Table 1. Institute Evaluation Summary 



Category 


Rating 


Appropriateness of meeting rooms and sizes 


I2-Excellent; 4-Above Avg 


Overall atmosphere 


I6-Excellent 


Lunch and refreshments 


I2-Excellent; 3-Above Avg; 
2-Avg 


Number of events and activities 

(please indicate if it was too much or not enough) 


I2-Excellent; 4-Above Avg 


Overall quality of the Institute 


I4-Excellent; 2-Above Avg 



Scale: Excellent, Above Avg, Avg, Below Avg, Poor; Responses: 16 out of 18 



In the two weeks immediately following the Institute, each committee convened to debrief their experiences. 
Minutes from each of these meetings were summarized and presented to the original planning committee. 

Suggestions and comments from faculty, as well as from Academic Technology professional and administrative staff, 
were examined. From this information, we drew a list of decisions or procedures that we will consider changing, 
adding, or avoiding in planning for next year’s Institute. Some representative items are: 

■ Move the Institute forward a week or two to avoid conflict with a Teaching Excellence program; 

■ Begin the application and review process 3 months earlier; 

■ Minimize the number of outside speakers and carefully vet their presentations; 

■ More actively involve the ATLs in the development and delivery of workshops; 

■ Devote more time to orienting faculty to their new laptops. 
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In addition to this structured evaluation, we also solicited more thoughtful feedback on the appropriateness 
of the specific objectives of the Institute through a follow-up survey and personal contact with participants 
beginning two weeks after the close of the Institute. 



Outcomes 

Thanks in large part to a systematic adherence to the ADDIE model, our first Institute was an overwhelming 
success. Planning is already underway for FITSI 2003. Interest and enthusiasm among faculty, department Chairs, 
Deans, and senior administrators is high. Academic Technology professional staff are already gearing up for the 
effort. Our short-term objectives have thus been met. 

In the larger scheme of things, we hope the Institute will engage a growing number of faculty in the theory 
and practice of instructional technology. In order to facilitate dialogue about instructional technology for a campus- 
wide audience, we will adopt an holistic approach to faculty development, one that takes into account the primacy of 
pedagogic concerns, the expressed needs of the faculty, and the collective knowledge and experience of our 
instructional technology professionals. Doubtless we will analyze, design, develop, implement, and evaluate a 
number of different strategies. Sometimes we will fail; sometimes we will succeed. 

Some of the projects we are currently developing include: 

■ An Educational Technology Assistant Program (eTAP) that will bring student assistants, AT 
professionals, and faculty together to form instructional technology teams to plan, develop, and 
implement course redesign through the use of instructional technology tools and techniques; 

■ An Instructional Technology Fellowship that will provide incentives for interested faculty to 
collaborate with AT professional staff to design and implement faculty development initiatives that 
relate new technologies to specific instructional objectives; 

■ A regular program of informal workshops and seminars that present and discuss “best practices’’ to 
guide faculty as they begin to think about a more robust use of technology; 

■ A Transforming Teaching through Technology (T^) award that will recognize exemplary efforts by 
individual faculty to use innovative technologies to improve their teaching. 

We will doubtless rely on the ADDIE model as we proceed with these initiatives. 

Our immediate objective will be to minimize the barriers that faculty encounter as they begin to integrate 
technology into their teaching. Our long-term objective will be to help faculty gain a deeper understanding of their 
own technology needs. Ultimately, we seek to foster a campus environment of development and growth that will 
underwrite the instructional technology vision we have for the campus. 
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Abstract 

In response to school-needs, state-requirements, and a PT3 grant from the Department of Education, the University 
of Northern Colorado is redesigning the two requisite educational technology courses for the undergraduate 
teacher preparation programs. The first course redesign has been implemented; the program's one-credit 200-level 
course is currently equipping UNC teacher candidates with basic computer and software skills. The second 
requisite course, the 300-level, has been redesigned, developed, and is now in the first stage of implementation. 

The first part of the course instructs teacher candidates in the concepts of technology integration. Then, to make 
those concepts more meaningful and transferable, the course immerses students in a case-based learning unit. This 
paper describes the unit’s problem scenarios. It also details the electronic environment and graphical user 
interface in which the scenarios and the information necessary for problem solving are presented. 

Introduction 

In 2000, UNC was awarded a Preparing Tomorrow’s Teachers to use Technology (PT3) grant. A major 
initiative of the grant is to redesign the educational technology undergraduate courses so that UNC’s School for the 
Study of Teaching and Teacher Education (STTE) is better able to prepare students to meet the new standards for 
using technology in instruction. The first year of the grant, the university’s PT3 team made up of the Project 
Director, Project Manager, and seven doctoral students from the department of Educational Technology (ET), 
redesigned the 200-level courses. Technology in Education, (i.e., the first educational technology course students 
take). The PT3 team crafted these courses to give students the technology tools they will need for superior 
performance throughout their university careers, for successful synthesis of technology integration theories with 
other learning theory, and, ultimately, for effective classroom instruction. The courses’ curricula vary for elementary, 
middle grades, and secondary preservice teachers. Currently, UNC’s undergraduate Preservice Teacher Education 
Program (PTEP) requires students to take two educational technology courses, with each course being worth only 
one credit. The preservice teachers generally take the first course (Technology in Education - 200-level) at the 
beginning of their teacher preparation and the second course (Educational Technology Applications - 300-level) just 
before their student teaching experiences. 

The 200-level curriculum includes introductions to both MAC and PC operating systems and introduces 
students to a variety of software (Word, Excel, PowerPoint, HyperStudio, Inspiration, and Dreamweaver) as well as 
how these tools can be integrated into a classroom setting. The second requisite course (300-level) has also been 
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recently redesigned to instruct preservice teachers in the theories of technology integration, classroom management, 
room design and software evaluation (Charsky, Ferguson-Pabst, Javeri, Liu, Sanzone, & Caffarella, 2003). Once this 
foundation is laid, the class then immerses the students in a case-based study where they must collect, analyze, 
process, and synthesize authentic information critical to solving the real-world problem. 

A study by Persichitte, Tharp, and Caffarella (1999) found that while technology is readily available to 
teacher education programs, only 45% of the programs’ faculty regularly use technology during class and only 40% 
of those programs require preservice teacher candidates to use technology to design and deliver instruction. Those 
statistics exist despite current standards for technology-use in education. Moursund & Bielefeldt (1999) suggest that 
until we will see reform in education programs, technology integration in K-12 schools will continue to lag behind. 
Further, research has shown that teacher preparation programs are not providing the type of training and modeling 
preservice teachers need in order to be proficient and comfortable integrating technology (ISTE). Moss (1988) reports 
that teacher education programs need to provide educational technology in order to better prepare prospective 
teacher. 

In 1999 the Colorado legislature passed Senate Bill 99-154, which includes eight standards for teacher 
education candidates to meet. The seventh standard states, “The teacher is skilled in technology and is 
knowledgeable about using technology to support instruction and enhance student learning.’’ However, a survey of 
the University of Northern Colorado’s (UNC’s) partner schools (i.e., local K-12 schools) found a high need for 
successful implementation of that standard. In fact, many schools rated teacher technology skills as their highest 
need. Clearly, professional and potential teachers across the educational occupation have not been equipped to meet 
the demand for trained teachers who can integrate technology throughout their instruction. 

Case-Based Learning 

Case-based learning environments generally incorporate components from several learning outcomes: 
simulations. Anchored Instruction (AI), Cognitive Apprenticeships (CA), and Problem-based learning (PBL). 
Simulations are defined as a series of sequenced activities which represent a social situation reduced to manageable 
proportions to serve a specific purpose. The Cognitive and Technology Group at Vanderbilt (CTGV, 1992) defines 
Anchored Instruction as an approach to help students become more actively engaged in learning. This type of 
activity is designed to cause thoughtful engagement that helps students, in a collaborative atmosphere, develop 
effective thinking skills and attitudes that support effective problem solving and critical thinking. A Cognitive 
Apprenticeship is defined as a methodology that provides a framework for students to be placed into authentic 
environments where they are asked to analyze content and develop appropriate strategies through social interaction 
in a way similar to that in real life craft apprenticeships (Brown, Collins, & Duguid, 1989; Casey, 1996; Brill, Kim, & 
Galloway, 2001). Levine (2001) defines problem-based learning as both a curriculum development and delivery system 
that allows students to develop problem solving skills while acquiring content knowledge and skills. Students take an 
active role in the learning process while acquiring knowledge in a problem solving, authentic environment (Jacobson 
& Spiro, 1995; Koschmann, 1995; Lajoie, 1993). Therefore, for the purpose of this project, the PT3 team chose to 
define case-based learning as a way to place students in an authentic environment where they are actively engaged 
in collaborative atmosphere thinking critically about creative ways to solve problems. 

Constructivist environments, like those of case-based learning, work well when students confront 
contextualized, ill-structured problems and strive to find meaningful solutions. Learners in this type of learning 
environment need to develop cognitive flexibility in order to transfer their knowledge to other situations and varying 
problems (Spiro et al., 1992). Constructivist and especially case-based learning environments permit such 
development. A case-based learning environment immerses students in a rich problem solving environment where 
they can, given the appropriate information, make the connections necessary for understanding the importance of 
technology integration and how such integration might look in a variety of classroom settings . 

The 300-level course. Educational Technology Applications, concentrate on preparing preservice teachers 
to integrate the tools learned in the 200-level class and other technology into instruction. The PT3 team wanted the 
300-level courses to give preservice teachers a picture of how integration looks in K-12 classrooms and ways in 
which they can use technology integration to meet state standards and solve instructional problems. After 
determining the course objectives, the PT3 team split into sub-teams for designing and developing the different parts 
of this course. 
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The first half of the 300-level course provides theoretical information about such topics as instructional 
design, software evaluation, classroom management, and teaching strategies. Although assignments throughout the 
course require preservice teachers to apply the concepts they learn, the PT3 team felt that situating knowledge and 
application in a more authentic, problem-based environment would help the PTEP teachers better understand all that 
technology integration involves. Therefore, the team decided the second half of the 300-level course should be a 
case-based learning scenario that immerses preservice teachers in a rich environment of active learning (REALs). 

As is characteristic of all REALs, the 300-level course’s environment relies on authenticity as a student- 
motivator. The environment’s authenticity also gives students the opportunity to organize concepts in such a way 
that they can transfer them to situations they may find in practice (Grabinger, 1996). The PT3 team talked with the 
university’s teacher education professors and K-12 teachers to devise weekly scenarios enriched by reality-based 
electronic artifacts (e.g., email notes fromthe principal and other teachers, phone messages in the form of audio-files, 
to do lists, letters, streaming video). The team wanted the case-based learning unit’s scenarios and materials to 
imitate, as closely as possible, typical experiences a first-year teacher might encounter. 

The active learning problems set forth in the 300-level course specifically meet the criteria for case-based 
learning and instruction. Ertmer and Russel (1995) write, “Case based instruction is a teaching method which requires 
students to actively participate in real or hypothetical problem situations, reflecting the kind of experiences naturally 
encountered in the discipline under study.’’ Students actively apply learned knowledge and skills to a problem 
situation. Ertmer describes these cases to be “ambiguous, messy, and recalcitrant.’’ Such a problem lies at the heart 
of the 300-level REAL. In fact, students must first analyze a fictitious school’s Colorado Student Assessment 
Program (CSAP) scores in order to define the case’s problem. The unit’s Web site 

(http://www.coe.unco.edu/ET34z/mnvc/) then offers the resources and information necessary for solving the 
problems . However, the 300-level course instructors also model ways in which the problems can be solved and act as 
technical and organizational support agents . 

Throughout the unit’s design and development, the PT3 team made a priority of building instruction that 
addresses the technology standards for teacher education as presented by the National Council for the Accreditation 
of Teacher Education Standards ( NCATE, 2001) and the state of Colorado. Each piece of the case-based learning 
unit enhances students’ understandings of technology integration and is directly linked to the standards. 

Graphical User Interface 

Upon entering the Web-based environment, the preservice teachers do not find the menus, navigation bars 
or buttons typical of traditional Web sites. Instead, they step into their new offices with the help of a unique 
graphical user interface (GUI) created by Caroline Sanzone, a member of the PT3 team and doctorate student in the 
Department of Educational Technology. Materials, in the form of email and telephone voice messages, letters, 
memos, and face-to-face meetings, that support the learning unit, are stored in and accessed via five active zones: 
the computer, telephone, in-box, file cabinet, and daily planner. 

The daily planner is the main user-cueing device; the planner is the user’s ‘to-do’ list. The list reminds 
users of the week’s important tasks, such as checking e-mail, and attending events (e.g., an in-service day focusing 
on CSAP analysis). A mouse-click on the appropriate active zone opens a small window and allows the preservice 
teacher to check email, listen to voicemail messages, read letters, or investigate the contents of files and folders. The 
windows, which users can drag anywhere on the screen and open simultaneously, give users the ability to customize 
their work areas. 

The complex design of the user interface is balanced by the authenticity of its elements. The office 
metaphor is not only relevant, but familiar as well. Anyone who has ever read a letter, opened an email, or used an 
answering machine possesses the prerequisite skills for successfully navigating the interface. Erom the first week of 
the unit to the last, the pre -service teachers see this case-based learning environment evolve, as new scenarios and 
pieces of information present themselves. As a result, the interface is flexible enough to accommodate this evolution, 
without forcing users to re -orient themselves every week. Just as one might save an old email or voice message for 
further review, the preservice teachers have access to items introduced in previous weeks, as well as new items , 
through the same five active zones. 

The unit’s scenarios are designed to realistically mimic a teacher’s work environment, including interactions 
with fellow teachers, administrators, and support staff. A mixture of audio, video. Acrobat, word processing, and 
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HTML files make these interactions seem more realistic. Video clips and sound files let the user experience the 
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Figure 1. Graphic User Interface 

The Scenarios 

The case-based learning experience evolves from the initial pretense that a Colorado school (Spring Hill 
Elementary) has hired a new teacher by the name of Miss Nelson. Miss Nelson’s specific grade level assignment is 
dependent on whether the PTEP students in the course are focusing on an elementary, middle, or secondary expertise 
level. The preservice teachers are asked to step into the role of this first year teacher (Miss Nelson) in order to solve 
the problems as they are presented week by week. The unit asks students to imagine that they have worked all 
summer to plan for their classes. However, much of that work is undermined when, in Week One, the teacher (Miss 
Nelson) receives a voice mail (in the form of an audio -file embedded in the unit’ s graphical interface) from the district 
Director of Personnel informing her that she has been reassigned to another school, this one with below average 
standardized-test scores. Week One of the case based learning unit exposes the preservice teachers to data, in the 
form of school profiles and standardized-test scores, that acquaints them with the new school’s environment. Week 
One’s GUI provides a letter, email message, newspaper articles, school profile, and various CSAP documents. The 
letter announces that Miss Nelson has received a grant from the Association for Technology and Learning (ATL) 
that will provide her classroom the funding for six computers, a printer, a scanner, a TV, a VCR, a digital camera, and 
$200 in discretionary funds. Miss Nelson also receives an email message from the district technology coordinator 
informing her that he has set up their computers, network connections, and peripherals and installed the necessary 
productivity software. Attached to the message is a PDE file which shows the floor plan of her room. The filing 
cabinet, in Week One, contains an archive of news articles from the Denver Post on varied CSAP issues, a detailed 
profile of Spring Hill Elementary School, and a copy of the past year’s CSAP reading scores. The GUI will eventually 
link to electronic cumulative files which contain report cards. Individual Education Plans (lEPs), reading proficiencies, 
and other student related documents . In addition to familiarizing students with the simulated school environment. 
Week One also familiarizes the learners with the unit’s online environment. Although each week’s interface is self- 
contained and intuitive, in Week One, the course instructors model standard procedures and point out the unique 
features of the interface. 

Week Two’s GUI first points Miss Nelson toward an email from the school principal, Ms. Trunchbowl, 
announcing an in-service and orientation meeting on CSAP analysis. In this message, she asks Miss Nelson to work 
with her grade level teaching partners to analyze the instructional needs of their classes. She and the rest of her team 
are asked to use the forms attached to the message to analyze individual student CSAP scores. Within this message 
is a link to a tutorial that explains how to interpret CSAP data and use the numbers to identify specific student needs. 
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During Week 3, things begin to get out of hand for Miss Nelson. The technology resources she received 
from the ATL grant have become a source of behavioral problems, distractions, and accidents. While working at the 
computers, students have been off -task and disruptive; they have not completed their work; and they have been 
crowding and rough-housing around the computers. Miss Nelson files both an accident report for a student who has 
been hurt when she trips over a power chord and a discipline referral for a student who is using the Internet 
inappropriately. These two referrals pose the third week’s problems. To combat the problems and make the best use 
of the equipment, the principal asks Miss Nelson to create a management plan that addresses safety and behavior 
issues, as well as detail a list of classroom rules for technology-use. The PTEP students also have the option of 
rearranging the classroom floor plan in a way that will reduce the problems being seen. To support the performance 
of those tasks, the GUI makes several resources available (i.e. a link that describes appropriate ways to use the 
Internet appropriately in the classroom, a newspaper article detailing Tech Troopers, and support materials from a co- 
worker offering job aids, tutorials, and posters.) 

In Week Four, after the preservice teachers are well acquainted with their environments, they are asked to 
design instruction for their (Miss Nelson’s) students. Week Four’s task is to revise a preexisting lesson plan so that 
the lesson meets the instructional needs determined by Week One’s data analyses. Preservice teachers are able to 
access lesson plans in the filing cabinet and then modify one of them to use the classrooms’ technology resources to 
enhance instruction. In Week Four, the GUI provides access to a database of lesson plans (i.e., the fictitious lesson 
plan bank that Miss Nelson spent her summer creating) that do not use technology. Miss Nelson receives a memo 
from a district curriculum official explaining she must begin implementing technology standards. This memo contains 
a copy of the CDF and National Education Technology Standards for Students (NETS). She also receives a letter 
from the ATL, reminding her to spend the discretionary funds she received on appropriate instructional software. 
Finally, during Week Four Miss Nelson receives a voice mail message from a parent expressing concern over the way 
computers are used in elementary classrooms. 

Week Five is framed as a teacher workday on technology integration. A memo from the school’s principal 
reminds Miss Nelson that all teachers will be expected to share their newly created lesson plans and give productive 
feedback. This memo includes an enclosure, the collaboration worksheet. Miss Nelson and her teaching partners will 
use to critique each other’s work. Miss Nelson, during Week Five, also receives a voice mail from a parent who has 
concerns about the age of their home computer. This mother is asking Miss Nelson to thoughtfully consider the 
homework she assigns that will require the use of a computer. 

During week six. Miss Nelson receives an email message from the school principal letting her know how 
pleased she is with Miss Nelson’s accomplishments and work with the ATL grant. Ms. Trunchbowl’s message has 
an attachment that provides information on grant writing. Miss Nelson also receives a letter from the grant-giving 
foundation (ATL), asking her to submit a report on how she has used the resources that the foundation provided. In 
this report, the preservice teachers must reflect on how they can use technology as a tool to support their teaching 
strategies. Week Six’s GUI also offers online resources for solving problems that inevitably occur when technology 
is used in classrooms . During week six’s class, the instructors give each preservice teacher a ‘trouble card’ that 
plagues their lessons with technical difficulties (troubles will be individualized so as to disable the delivery of each 
teacher’s lesson). In response, teachers must create contingency plans (e.g., handouts, library research of print 
materials) for their lessons. In the unit’s final week. Week Seven, the PTEP students discuss their experiences, 
present their lesson plans, and submit their final report to the grant foundation. 

Timeline 

The PT3 team has designed and developed the case-based learning unit discussed in this article over the 
past year. The implementation of the course began in September, 2002. Data collection instruments such as the 
Stages of Concern Questionnaire (SOCQ), Attitudes Toward Computer Use Survey (ATCUS), instructor 
observations, student exit interviews, and a Likert scale survey on course effectiveness will assist the team in 
gathering data about the courses’ effectiveness. The SOCQ measures how concerns about different innovations, in 
this case, technology integration, change over time (Horde & Hall, 2001); the ATCUS measures an individual's 
attitude toward computer usage (Popovich, Hyde, Zakrajsek, & Blumer, 1987); Instructor observations, Likert scale 
survey, and student exit interviews will add anecdotal information on student’s projects, interaction with the 
interface, learning behaviors, and the general course effectiveness. Currently, students are administered the SQCQ 
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and ATCUS in both the 200- and 300-level courses, and, after implementation, the team will continue to administer 
these instruments twice each term. The team will also analyze student projects as qualitative data that will indicate 
whether the case-based learning unit is effective in meeting instructional objectives. 

With the importance of training teachers to effectively integrate technology, case based methods may be 
one of the most powerful tools one can utilize to meet state and national standards regarding this goal. Because 
teacher education students often do not see effective technology integration modeled in either their student teaching 
experiences, or in the majority of their college courses, one class often may not be enough to present the ill defined 
domain of technology integration to preservice teacher candidates. This case study may prove to provide enough 
information to give learners a basic understanding of what technology integration not only looks like but how it can 
be used to solve problems and help to meet state and national education standards. Activities such as these will also 
add to the body of knowledge regarding the effectiveness of constructivist environments, in particular, case based 
learning environments. 
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Abstract 

The Department of Educational Technology PT3 team at the University of Northern Colorado has worked with six 
cohorts of faculty fellows over the past two years in order to redesign thirty-two courses. The process the team 
follows has evolved over the past two years in ways that better support the faculty and their coaches. This article 
will document the methods used to work successfully with faculty cohorts and discuss selected projects, members of 
the team have supported. 

Introduction 



Grant Overview 

In June of 2000, the Department of Educational Technology at the University of Northern Colorado (UNC) 
received a PT3 grant (Preparing Tomorrow’s Teachers to Use Technology) funded by the United States Department 
of Education. The grant, Infusing Technology Use in the Preparation of Colorado Preservice Teachers, sets forth a 
blueprint for systematically changing the teacher education programs at UNC by infusing technology throughout the 
curriculum. The changes affect Pre-service Teacher Education Program (PTEP) students during their entire program 
from initial admission through student teaching. The project has three overarching goals: (1) graduates of the teacher 
education programs will effectively utilize technology for instruction in their classrooms when employed as full-time 
teachers; (2) student teachers will effectively utilize technology for instruction in the partner school classrooms; and 
(3) faculty members will effectively utilize technology for instruction and model appropriate technology use for the 
preservice teacher education students. 

The PT3 grant team has worked toward these outcomes through five initiatives, two of which deal 
specifically with UNC faculty. Initiative Two calls for Professional Teacher Education Program (PTEP) faculty to 
model appropriate technology use and integrate technology utilization into their courses. Initiative Three proposes 
the same for general education and content area discipline faculty. Currently, the grant team has worked with six 
cohorts of UNC faculty who have learned about and applied technology in new ways in their undergraduate 
preservice and general education courses. Thirty -two UNC faculty fellows, 15 from the College of Education (COE) 
and 17 from the College of Arts and Sciences (A&S), have redesigned at least one of their courses to effectively 
utilize and seamlessly integrate technology. 

Background 

Although there has been a major investment in technology in K-12 and higher education institutions, the 
actual utilization and integration of this technology have been somewhat disappointing. Research by the Office of 
Technology Assessment (1995) and others (Moursund & Bielefelct, 1999) indicate that without reform to the 
education of teachers we won’t see improvements in technology integration in the general education curriculum. 
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Higher education faculty members are beginning to be held more accountable for the use of technology and 
its integration into their coursework than in the past (Cheung, 1999). The students, however, are not reporting an 
increase in the use of technology in their general education or content area courses for instructional delivery, with 
assignments or student communication. Preservice teachers need to see the effective use of technology modeled so 
they will be prepared to use technology in their own classrooms. If preservice teachers do not experience the 
effective use of technology throughout their academic career, both in education and content area courses, they will 
not have models upon which to build their own personal views of technology integration. 

Most preservice teachers do not see good modeling of technology integration or effective instructional 
strategies in their undergraduate instruction (Bruder, 1993). Machanic (2001) found that only limited faculty 
development is being done in relation to technology rich environments. In order for preservice teachers to use 
technology effectively, colleges of education must increase the opportunities for undergraduates to see technology 
modeled and use technology effectively throughout their educational career (Fawson & Smellie, 1990). In 1997 and 
1999, Persichitte, Tharp, and Caffarella completed studies commissioned by the American Association of Colleges of 
Teacher Education, which investigated student and faculty use of technology by schools, colleges, and departments 
of education (SCDE). The authors found that the 744 institutions surveyed generally had extensive infrastructure 
already in place, but the faculty were either not using it at all, or they were only using it for personal productivity, 
rather than for instructional activities. The data als o showed that faculty members were not requiring students to use 
technology for class projects. Both of these studies emphasized the importance of preservice teacher education 
students’ exposure to effective models of technology use and integration throughout their preservice preparation 
programs. 

In response to senate Bill 99-154, the Colorado State Board of Education enacted eight standards for 
licensing of teacher education candidates. The seventh standard is, “Knowledge of Technology” and is defined as, 
“The teacher is skilled in technology and is knowledgeable about using technology to support instruction and 
enhance student learning.” Current assessment of the technology competencies of the PTEP students is that they 
have strong educational technology skills once they complete the two required undergraduate courses in 
Educational Technology. Students report, however, that there is still limited use of technology in their coursework. 
Eor the PTEP students to be thoroughly prepared to use technology in the classroom they must experience its 
effective integration into their coursework. By experiencing the effective integration of technology in education, 
general education, and content area courses, the pre -service students will be given valuable models upon which to 
build their own technology integration skills. 

Process 

Each term, faculty is invited to apply to participate in the grant by going through a Request for Proposal 
(REP) process. The online application (http://www.coe.unco.edu/PT3/pt3Application.doc) asks faculty members to 
complete a project abstract, detail demographic data (e.g. name, department, course title and description) and write a 
three to five page project proposal that includes the proposed uses of technology, how the proposed uses will model 
technology for the preservice teachers, how the proposed technology will improve the course, and the proposed cost 
of supplies, software, and materials necessary for the redesign efforts. This proposal needs to detail both the delivery 
of instruction and student use of technology. The proposals must also include documentation showing the number 
of PTEP students that typically take the course. The grant requires a minimum of 50 PTEP students per semester per 
class or that the class has at least 75% PTEP students. Eaculty members who apply to work with the grant do not 
need any prior technology experience other than the ability to complete the REP in a word template and attach the 
completed document to an email message. 

The project team, the COE Dean, and the A&S Dean all work together to review the proposals. Each term, 
the team selects between six to eight faculty members to participate as fellows. Eaculty Eellows who are accepted are 
then released from one course for a semester, given a $500 stipend to purchase materials necessary to support the 
course redesign, and assigned an educational technology graduate student who serves as a coach for the term. The 
first faculty members were selected in the fall of 2000 and released from one course during the spring of 2001. Eive 
subsequent cohorts of faculty fellows have worked with the team during the summer of 2001, fall of 2001, spring of 
2002, summer of 2002, and fall of 2002. 
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Faculty /Student Coaching Teams 

Each faculty member accepted to work with the grant is paired with an advanced doctorate student from the 
Department of Educational Technology who serves as a technology coach. Once Eaculty Eellows are selected, the 
team matches, as closely as possible, the interests and proposal descriptions of the faculty to the backgrounds and 
expertise of the student coaches. At the beginning of each term, coaches begin the working relationship with their 
faculty member by contacting them and arranging an initial meeting to discuss the ideas presented in the faculty 
member’s proposal. During this initial meeting, the team develops potential technology integration strategies, 
identifies the project’s goals, prioritizes the tasks required to complete the project, considers any constraints of the 
project, sets regular weekly meeting times, creates a semester long schedule of activities necessary for the success of 
the project, and defines a timeline in which to complete the activities. Coaches provide the training, support, 
appropriate technology integration tips, as well as instructional design expertise so each faculty member is able to 
accomplish their goals . Together, the student/faculty team conducts a needs assessment and audience anlaysis for 
the course and identifies appropriate places to infuse technology. Once the assessment is completed, the 
faculty /coach team selects, designs, and/or produces appropriate instructional materials for the class. Because of the 
focus of the grant’s overarching goals, the team has stressed the importance of the coaches supporting faculty in a 
way that they become independent and self sufficient with their usage. Our goal has not been the completion of 
projects as much as the support of systemic change within the institution. Eaculty Eellows come to the project with a 
wide variety of expertise, progress at various speeds, and complete redesign efforts with varying success. Each 
chooses different areas in which to focus, depending on their particular interests and ideas for change. Eaculty 
Eellows have worked with electronic portfolios for assessment, synchronous and asynchronous communication, 
teleconferencing, web-based learning environments, multimedia project-based learning activities, adaptive 
technologies, productivity software, inquiry learning, simulations, online course delivery, and video production. 

Access to adequate software, hardware, and networking is critical to the redesign efforts. The 
faculty/student teams are fortunate to have access to high-end computer technologies in several COE computer 
laboratories. Eaculty may choose from Smart Classrooms, as well as complete laboratories in either MAC or Windows 
platforms. Projectors, scanners, and digital still and video cameras are also available for their use. Once faculty 
members have completed their semester with the grant, they have diverse support options through the university in 
order to continually modify their courses. Support includes, but is not limited to, computer lab consultants. 
Informational Technology Services, and COE and A(&S technical support personnel. In addition, some of the 
faculty/student teams have continued their collaborative efforts after the fellowship has been completed. The 
redesign efforts have resulted in curriculum revisions that are both instructionally improved and technologically 
enhanced. 

After having worked with several cohorts of faculty members, the UNC PT3 team came to the conclusion 
that additional structure was necessary in the process of working with Eaculty Eellows. An online hypertext packet of 
materials ( http://www.coe.unco.edu/PT3/Facultv/Packet/Packet.htrri) was created to detail expectations of faculty and 
limit problems encountered with past cohorts. This packet includes the topics of the weekly seminars as well as 
forms to complete during the term: GA/Faculty Fellows checklist of expectations, project goals, prioritized task list, 
timeline of activities, monthly reports, and final project report. All of these items are returned to the Project Director or 
Manager in digital form in order to keep complete records for federal reports. The student/faculty teams review the 
original proposal, identify achievable goals, decide of project restraints, equitably divide responsibilities, identify 
actual faculty needs, and produce monthly and final reports. Adding this structure has eased the fear factor faculty 
experienced, improved the redesign process, allowed the process to run more smoothly, and markedly enhanced the 
end products. 

Faculty Fellow Seminars 

All of the faculty members chosen to participate in initiatives two and three during a given semester also 
participate in faculty development seminars on technology use in the classroom. The weekly seminars provide the 
opportunity for Faculty Fellows to extend their knowledge and understanding of technology use in instructional 
settings and realize the importance of embedding the technology use within their courses. Professional development 
in the area of technology integration must carefully avoid over emphasizing the bells and whistles of the technology 
so that faculty members focus their attention on the message, not the technology used to deliver it (Machanic, 2001). 
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The Project Director of the grant leads the faculty development seminars with the support from the Project 
Manager and other faculty members from the department of Educational Technology. The content of the seminars is 
based on the needs and interests of the individual faculty members during each semester. The contents of the 
seminar presentations have included: exemplar redesign efforts, effective PowerPoint presentations, advanced word 
processing techniques, Dreamweaver basics. Smart Classrooms, Blackboard training, mechanics and conventions of 
the web, synchronous and asynchronous communication, Web-based learning environments, creation and 
manipulation of digital graphics, electronic portfolios, file management, student management systems, audio and 
video production, and technology integration. These seminars, approximately 16 hours over the course of the 
semester, provide an opportunity for Faculty Fellows to not only gain knowledge of new technologies, but also share 
successes, concerns, and frustrations in an environment of support and problem solving. The Project Director, Grant 
Manager, and Educational Technology faculty members, provide structure to and content for the seminars, identify 
subject matter experts to deliver further training, and provide a link between the Faculty Fellows, graduate coaches, 
and content explored during the seminars. 

Showcase of Projects 

Because each one of the faculty fellows has taken a slightly different approach to their redesign effort, 
depending on the outcomes identified in their proposal as well as their personal skills level, the final products have 
varied greatly . Examples of Faculty Fellow projects can be found on UNCs PT3 website 

(http://www.coe.unco.edu/PT3). Through interviews with faculty fellows after their redesign efforts have been 
completed the PT3 team has learned a great deal about the benefits of this type of staff development project. Faculty 
Fellows have shared the following comments: 

- Working with PT3 in my redesign of the class was a lot of work; however, the skills, knowledge, and 
competence gained by my students outweighed the hard work I did and will continue to put in. 

- This was a very fascinating and stimulating experience for me. It allowed me to use current technology more 
effectively while learning how to incorporate new types of technology into my courses. 

- The program was especially useful to me because it helped me restructure and improve my course in ways I 
hadn’t originally planned. My graduate coach helped me overcome the steep learning curve associated with 
the use of the new web page creation software. 

- The PT3 fellowship has been a great experience for me. There were several technology related projects I 
wanted to incorporate into my teaching, but frankly I had no idea how to accomplish them. Working with my 
graduate assistant and the other PT3 staff made what seemed impossible, possible. 

- Working with the PT3 grant has been a most extraordinary experience. I have been intrigued by the 
applications I have seen colleagues and students use. The seminars allowed me to become familiar with 
some and even proficient with others. 

- I discovered that I could really make and record video clips on CD and the process wasn’t too difficult. This 
led me to be able to require this of my students which will help them illustrate their own lessons for their 
classes. 

- The most rewarding aspect of my work was becoming comfortable in the use of DreamWeaver. Not only did 
I create what I think is an attractive and user friendly website, I have become confident in my ability to 
manipulate the program to improve the website and work with students on their websites. This success is 
due in no small part to the second most rewarding aspect of the work, the professional relationship I 
developed with my technology coach. He was at all times professional, helpful and available. 

- My main idealistic goal for this project was to develop such a familiarity with the technology that I would be 
comfortable in imparting this familiarity to my students. I wanted to be able to integrate technology into my 
understanding and presentation of history content. If the integration becomes seamless, and my students 
see it as seamless, than they in turn will be able to impart this technology onto their students in the future. 
The goal is to make technology an integral part of history pedagogy, for me and for the future K-12 history 
students that they teach. I am not sure that this goal will be realized, but at this point I suppose it is enough 
to say that I am confident enough to make the attempt. 

- I most enjoyed having the time and support to make real some of the ideas I have had about my class for a 
long time. Without the one-on-one training and support, I probably never would have figured out how to 
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do something like this on my own. It was great to have a coach who was willing to sit down and talk me 
through the process in a step-by-step manner, but who also made sure that I was able to do things on my 
own (so that I could continue the work solo once the semester ended). It was also rewarding to hear and see 
what other fellows (both in my own group and from previous semesters) were working on or had already 
done, as well as to be exposed to other kinds of technology that I never would have considered before (e.g., 
sound editing). Those opportunities helped me start thinking about other possible ways of incorporating 
previously unknown kinds of technology into my classes. I think this is a wonderful experience for any 
faculty member, regardless of their prior experience with technology. I consider myself fairly tuned into 
technology uses in the classroom, but I discovered that there was a whole world of possibilities out there 
that I hadn’t even considered. It’s a great way to become acquainted with the range of possibilities for 
future technology use and to discover that the skills for using those technologies are entirely learnable. 

- Working as a faculty fellow with the PT3 grant was time well spent! I learned and continue to learn much 
about ways to use technology in my online and campus classes. Since my involvement with the PT3 project 
I have also become more comfortable and committed to using technology, when appropriate, in my teaching. 

- The PT3 experience exposes faculty to cutting-edge techniques for course preparation and presentation. It 
has proven to be an invaluable experience and has enhanced not only the focal course, but all my teaching 
endeavors. 



Summary 

Through the support of the graduate assistant coaches. Project Director, Project Manager, and other subject 
matter experts, the Faculty Fellows are able to not only redesign the course for which they are chosen to participate 
in the grant, but continue experimenting and learning on their own. Enthusiasm has been high among the Faculty 
Fellows who have participated in this fellowship opportunity. Although they begin their journey at very different 
levels of proficiency, most have significantly improved their skills and ability to seamlessly integrate technology into 
the content they deliver. 

Fundamentally, the PT3 grant is based on the premise set forth by Kent & McNergney that “technology 
could transform the most basic tools used by teachers, and this transformation might also create radically new types 
of learning environments” (Kent & McNergney, 1999, p. 51). The long-term results described here for the redesign of 
PTEP, general education, and content area discipline courses are not yet known, but the short-term and systemic 
effects are already yielding great dividends. Eaculty Eellows who have participated in the grant and successfully 
integrated various technologies into their courses are excited about the results they have achieved. Current PTEP 
students are beginning to see technology modeled in their education and content area courses. The work made 
possible by the PT3 grant is allowing preservice teachers to see and value the use of technology for teaching and 
learning. Hopefully, the effective uses of technology in preservice teachers’ courses and the opportunity for them to 
develop their own skills will serve as models that UNC’s future teachers will employ one day in their own classrooms. 
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Microsoft Windows XP® Accessibility Features 

Tom At kin son 

Central Missouri State University 

In recent years, computer technology has dramatically changed the way we access information. And for no 
group is this more dramatic than for those with disabilities. With the support of assistive technology, people with all 
types of disabilities will be better equipped to enter the mainstream of information access. 

A person without the use of their hands can enter text by talking to a computer. 

A person with severe hearing loss can freely use the telephone. 

A person without sight can instruct a computer to read aloud everything on the screen. 

A young child with no voice can communicate through a talking computer. 

In the evolution of technology, this is a critical time in our society for people with disabilities who have so 
much to gain by accessing emerging technologies - and so much to lose if access is denied. According to the 
Alliance for Technology Access, while the incidence of disability in the general population is 20%, it is even higher 
in communities of mixed ethnicity and economically disadvantaged and in rural communities. Research indicates that 
67% of all adults with disabilities are unemployed. Disability (seen or unseen) is a part of every community and it is 
critical that everyone have equal opportunity to move toward a more accessible environment. (Alliance for 
Technology Access: tasc.ataccess.org) 

There are several types of assistive technology available to improve accessibility when using comp uters: 

Screen enlargers and screen magnifiers work like a magnifying glass by enlarging a portion of the screen as 
the user moves the focus. 

Voice input aids or speech recognition assist people who have difficulty using a mouse or keyboard. These 
aids allow users to control computers with their voice instead of a mouse or keyboard. 

Screen reviewers and screen readers make on-screen information available as synthesized speech or a 
refreshable Braille display. Generally, they can only translate information that is text. Graphics can be translated if 
there is alternative text describing the visual images. 

An on-screen keyboard can help those unable to use a standard keyboard select keys using a pointing 
method such as pointing devices, switches, or Morse-code input systems. 

Keyboard enhancement utilities help those with trouble typing— including increasing typing speed. 
Assistive technology can compensate for erratic motion, tremors, slow response time, and other related conditions. 
Word prediction utilities offer other types of keyboard filters including typing aids. 

Alternative input devices allow individuals to control their computers beyond the standard keyboard or 
mouse with eye-gaze pointing devices, sip-and-puff systems controlled by breathing, and non-standard keyboards. 
(Microsoft Corporation, www.microsoft.com/ ) 

Until recently, these enhancements required the user to obtain specialized software that provided the 
desired feature. With performance improvements in computers, accessibility features are being gradually 
incorporated into the operating system and are able to address the following types of impairments: 

Vision 

Hearing 

Mobility 

Cognitive 

Language 
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From negligible to extreme, the range of impairments is broad. Symptoms of low vision include dimness, 
haziness, extreme far-sightedness or near-sightedness, color blindness, and tunnel vision. Those with hearing 
impairments may be able to hear some sound, but may not be able to distinguish words. Others may not hear any 
sound. For those with hearing impairments, computer prompts such as beeps and spoken messages can be 
problematic. Users with hearing impairments need visual signals for all information otherwise conveyed by sound. 
Mobility impairments can be caused by a wide range of illnesses and accidents such as arthritis, stroke, cerebral 
palsy, Parkinson's disease, multiple sclerosis, loss of limbs or digits, repetitive stress injury, etc. Poor muscle control 
or weakness can make using standard keyboards and mouse devices difficult. Some people are unable to type two 
keys simultaneously, while others may hit multiple keys or repeat keys when pressing or releasing them. Those with 
use of only one hand experience difficulties with certain keyboard and mouse tasks. (Microsoft Corporation, 
www.microsoft.com/ i 

Cognitive and language impairments ranging from dyslexia to difficulties remembering, solving problems, or 
perceiving sensory information to problems comprehending and using language can make using computers more 
difficult. A language translator could prove beneficial to people with any of these disabilities. (Microsoft 
Corporation, www.microsoft.com/ ~) 

Accessibility Features 

Computer accessibility features provide better integration with assistive technology and easier 
communications that allow people with accessibility needs to work more effectively. The appearance and behavior of 
these features can be adjusted for specific vision, hearing, mobility, cognitive, and seizure-related needs. This article 
highlights many of the accessibility features included in the new Windows XP® operating system for PCs. 



All features are installed within the operating system but 
must be activated and configured before they can be used. 

Standard accessibility options change the appearance and behavior 
of the keyboard, display, sound and mouse functions through the 
Accessibility Wizard, the Accessibility Options icon, and other 
icons in the Control Panel. Mouse and cursor options adjust the 
motion, size, color, tracking, and rate of blinking and flashing for 
those with seizure disorders that are affected by timing patterns. 

Shortcut keys called “hot keys” immediately activate 
accessibility features to help those who cannot use the computer 
without first having accessibility features enabled. “Plot keys” 
substitute for select commands and buttons from menus and 
toolbars for people who have difficulty using the mouse as their 
standard navigation device. 




Figure 0. Accessibility 



Additional features include magnifier, narrator, and on-screen keyboard to support individuals who require 
temporarily using a computer other than their own, or to log on and set up their machine for the first time. A “Utility 
Manager” enables users to start and stop the magnifier, narrator, or on-screen keyboard application programs. 

While these utilities are not intended as replacements for full-featured assistive technology products, many of the 
utilities provide very useful features available on any computer using the Windows XP® operating system. 

The Accessibility Wizard provides an easy step-by-step approach for selecting or disabling specific 
features and configuring each option to user preferences. Microsoft Windows XP® menus are automatically 
updated to place frequently used selections at the top of the menu where they are usually more accessible. Disabling 
this feature may be preferable to some to allow a more consistent list of menu selections. To maximize the 
accessibility features, several taskbar and menu settings are recommended: 
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Ensure that the taskbar remains visible, even when running a program in a 
maximized (full-screen) window. 

Display the “Quick Launch” toolbar to display the Windows desktop or start a 
program with a single click and customize it with buttons to start favorite programs. 

Customize the Start menu by using large icons and specifying the number of 
programs you want to display on your Start menu. The size of icons and number of 
items on the Start menu can benefit those with vision impairments and 
cognitive/learning disorders 

Specify the “Classic Start Menu” for the familiar look and behavior used in 
earlier versions of Windows. 
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Figure 0. Start 



Following the initial configuration using the wizard, options may be changed on dialog boxes for modifying 
display, sound, keyboard, and mouse characteristics. 

Display Settings 

The display settings provide a variety of settings to control the way information is presented to the user. 
Normally, higher legibility of documents can be achieved by adjusting the screen resolution of the video adapter to a 
higher setting (i.e. 1024 X 768). However, higher resolution settings often decrease the size of objects on the screen. 

Objects can appear larger with lower resolution settings (i.e. 640 X 480) that benefit 
individuals with vision impairments. High Contrast color schemes can increase 
legibility for some users by heightening screen contrast with alternative color 
combinations. 

Some of the display schemes use font size to improve legibility. Users make 
objects on the screen more visible by selecting the option to enlarge icons. Normally, 
by pressing keys that correspond to the underlined letters in menus, commands, or 
dialog box options, one can quickly select options using the keyboard, rather than 
the mouse. These features can confuse or disguise characters on the display, 
reducing legibility. Select the option to hide underlined letters for keyboard 
navigation and input indicators (the dotted rectangles around objects) until keyboard 
navigation is accessed with the ALT, TAB or arrow keys. 

Windows XP® provides the “Magnifier” application software as a display 
utility for those who have low vision difficulties. The “Magnifier” provides a 
minimum level of functionality for people who have slight visual impairments by 
creating a separate window that displays an enlarged portion of the screen. Custom features include position, size, 
inverted colors, magnification level, and tracking options. 

Sound Settings 

The ability to clearly hear the sounds made by a computer is important to those who rely on sounds to 
acquire information from the computer — especially those who are blind or vision impaired. The volume of add-on 
computer speakers can be adjusted independently. A sound scheme can be applied containing settings that 
associate many program events to specific sounds (i.e. Close Window, Exit Windows, New Mail Notification, or 
Incoming Fax). Unique sounds associated with system events help those who rely on sounds to get information 
from their computers. 

Windows XP® can display visual warnings for system sounds or display 
captions for speech and sounds within programs. “SoundSentry” allows the user 
to generate a blinking title bar or flashing border/screen whenever the computer 
generates a sound. Visual warnings can indicate sounds made by windowed 
programs and for sounds made by full screen text programs. “ShowSounds” is an 
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Figure 0. Sound 





Figure 0. Display 





accessibility feature that instructs programs that usually convey information only by sound to also provide all 
information visually as text captions or informative icons. 
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Text-to-speech (TTS) provides playback of printed text as spoken words. An internal driver, called a TTS 
engine, recognizes and speaks written text through a synthesized voice selected from several pre -generated voices. 
The TTS engine is installed with the Windows XP® operating system. Additional engines are available from other 
manufacturers. 

“Narrator” is a text-to-speech utility for those who are blind or have 
low vision. “Narrator” reads what is displayed on the screen — the contents of 
the active window, menu options, or text that has been typed. It is designed to 
work with Notepad, WordPad, Control Panel programs, Internet Explorer, the 
Windows desktop, and some parts of Windows Setup. The Narrator application 
may not read words aloud correctly in other programs. “Narrator” may be 
customized by speed, volume, and pitch to announce events on the screen, read 
typed characters, and move the mouse pointer to the active item. 

Figure 0. Narrator 
Keyboard Settings 

Standard settings allow shortcuts to menu items by holding the “ALT” 
or “CTRL” key and pressing the corresponding key indicated by the underlined 
letter. Most system events may also be activated using shortcuts (“hot keys”). 

The amount of time that elapses before a character repeats when holding down a 
key, referred to as “repeat delay”, and the speed at which a character repeats 
when holding down a key, referred to as “repeat rate”, can be adjusted to slower 
rates to allow users more time to respond. To provide an audible cue, a click 
sound can be produced when a key is depressed. These actions can be set to 
toggle on/off with "hot keys." 

Microsoft Windows XP® provides several mobility options to assist 
those who have difficulty using the keyboard. The “On-Screen Keyboard” 
displays a virtual keyboard that allows those with mobility impairments to enter 
data using a pointing device or joystick. Besides providing a minimum level of 
functionality for those with mobility impairments, “On-Screen Keyboard” can 
also be of help to those who do not know how to type. 

The “On-Screen Keyboard” supports entering characters by several methods that include clicking, 
hovering, and scanning. In clicking mode, data can be typed by pointing the mouse cursor to the keyboard 
characters and clicking. In hovering mode, a mouse or joystick is used to point to a key for a predefined period of 
time to type the character automatically. In scanning mode, On-Screen Keyboard continually scans the keyboard 
and highlights areas where you can type keyboard characters by pressing a hot key or using a switch-input device. 

To make the keys on the “On-Screen Keyboard” more legible, select a large font style and size. If using a 
language other than U.S. English, it may be necessary to change font options to see additional characters. The 
selected font for the “On-Screen Keyboard” keys does not affect the font used in the selected program. To keep the 
“On-Screen Keyboard” displayed when switching to other programs or windows, select the “Always on Top” 
option. The “Click Sound” option adds an audible click when selecting a key on the “On-Screen Keyboard.” 

Other important keyboard accessibility features 
include “StickyKeys”, “FilterKeys”, and “ToggleKeys.” 
“StickyKeys” helps those with difficulty holding 
down two or more keys at a time. When a shortcut requires 
a key combination such as “CTRL-i-P”, “StickyKeys” will 
enable the user to press one key at a time instead of pressing 
them simultaneously. “FilterKeys” adjusts the keyboard 
response so that inadvertently repeated keystrokes are 
ignored. “FilterKeys” can slow the rate at which a key repeats when held down. “ToggleKeys” helps those with 




Figure 0. On-Screen Keyboard. 




Figure 0. Keyboard Settings. 
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vision impairment or cognitive disabilities by providing sound cues when the locking keys (CAPS LOCK, NUM 
LOCK, or SCROLL LOCK) are pressed. A high- pitched sound indicates the keys are switched on and a low pitched 
sound when keys are switched off. 

Mouse Settings 

A PC mouse typically supports two buttons designated as left and 
right. One button is used to position the cursor and to click and drag items. The 
other button displays a menu of tasks or options related to the item clicked. The 
button functions can be reversed if needed and the double-click response of the 
buttons can be adjusted faster or slower. Pointer speed relates to the distance 
that the pointer moves respective to the distance that the mouse or trackball 
moves. Changes to this setting can affect the user’s ability for precision of the 
pointer and movement across the screen. For example, to move the pointer 
across the width of the screen, the pointer device must move farther when set to 
slow than when it is set to fast. A “ClickLock” option allows the user to 
highlight or drag an item without holding the mouse button. A “SnapTo” feature 
moves the pointer automatically to the default button in a dialog box so it is 
Figure 0. Mouse Settings. easier to see which button or option has focus to be activated when the ENTER 

key is pressed. 

Pointer schemes contain predefined sets of mo use pointers, such as “Magnified” or “Windows Black (extra 
large)” that increase visibility of the mouse pointer on the screen. Eor better visibility when working with a 
document, especially when using an enlarged mouse pointer or a pointer scheme, the mo use pointer can be hidden 
while typing. The mouse pointer reappears when moving the mouse or trackball. Also, pointer trails can be added to 
help the user track pointer movement across the screen. This feature is especially useful when using Liquid Crystal 
Display (LCD) screens. To easily locate the mouse pointer, an option can be set to provide an animated visual cue 
by holding down the “CTRL” key. 

Eor those with difficulty using a mouse, Windows XP® provides an application called “MouseKeys” that 
uses the numeric keypad to control movement of the mouse pointer. To use the numeric keypad for data entry and 
navigation, activate MouseKeys by pressing the “NUM LOCK” key. 




General Settings 



Several accessibility options can be found under the General settings 
dialog window. The user can be warned either visually or audibly when 
accessibility changes occur. When this option is selected, a warning appears or a 
confirmation sound plays every time a shortcut key is used to turn an accessibility 
feature on or off. This is useful for alerting users who might turn on a feature 
accidentally. By incorporating these functions into the operating system, all 
applications can potentially support alternative inputs from single switch or puff 
and sip devices plugged into the computer's serial port. 

The accessibility preferences can be applied to other users when logging 
on to the system and can be set to automatically become disabled after a specified 
idle time. In this way, all users will be able to access features to help them log on 
to the computer. When multiple users share a computer, Windows XP® permits 
fast switching between users (without actually logging off from the computer) 
allowing multiple users to share a computer simultaneously without closing the 
programs they are running. 

The cursor is used to indicate a focus point for entering and selecting information. Locating and following 
the cursor can be difficult even for users without visual impairments. An option may be selected to automatically 
move the cursor when the focus changes. This option improves the functionality of some screen readers and screen 




Eisure 1. General Settinss. 
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magnifiers that use the cursor to determine which area of the screen to read or magnify. The blink rate and width of 
the cursor can be changed to make the cursor easier to see on the screen or less distracting for those with vision 
impairments. Adjusting these settings can benefit people with mobility impairments. Those with seizure disorders 
related to flashing or blinking objects may want to turn the cursor blinking off. 

Internet Accessibility 

One of the most valued resources a computer can provide is access to the Internet. While many Web sites 
are not very user friendly, efforts are being made to improve accessibility. Many barriers still exist for users with 
visual impairments. Internet Explorer® Version 6 that comes with Windows XP® provides several features that 
support greater accessibility when browsing the Internet. These features are modified by selecting “Tools - Internet 
Options” on the browser menu. 

To improve visibility, the browser can be set to ignore colors, font 
styles, and font sizes specified by Web authors and to use alternate settings 
selected by the user. If the author of a Web page chooses different colors 
or font styles, the default settings will override them using custom style 
sheets. To increase contrast between screen elements, text and background 
colors can be pre-selected for displaying Web pages on the screen. Also, a 
specific color may be selected to designate Web page links not visited and 
those already visited. Particular color choices make links easier to see for 
those using high-contrast color schemes. Color settings can be chosen for 
highlighting links when hovered by the mouse. 

Web designers often include alternative text information associated 
with an image on the page. Occasionally the description is cut off when 
using small images. To ensure that alternate text is not cut off if the amount of text is larger than the image area, the 
user should select the option to always expand the text. This option expands the image size to fit all of the alternate 
text. 

When reviewing Web pages, users often scroll back and forth through the information. Screen readers will 
sometimes read part of the next link and extraneous information along with the next link. To prevent this problem, the 
scrolling feature can be set to a predetermined speed that allows screen readers to continue to read links correctly 
even when the next link is off the current screen. This option often improves voice recognition programs as well. 

Animations, sounds and videos typically play automatically when displaying Web pages. Turning off the 
visual experience will help display pages more quickly. Those with low vision or sensitivity to screen flashing can 
eliminate the distractions by turning off this feature. 

Summary 

The Americans with Disabilities Act brought to the surface many of the deficiencies computer software 
producers must address regarding individuals with physical and cognitive impairments who use computers. 

Microsoft Windows XP® offers many accessibility features with options to adjust specific features to meet the 
needs of those requiring them. Although these features do not fully address the needs of all users with disabilities, 
they support an interface for other manufacturers and a common direction that will someday become available on all 
computers. 

Eventually, Web sites will include locations with more significant features for 
accessibility. The CPBAVGBH National Center for Accessible Media (NCAM) provides 
a symbol that may be used by webmasters to denote that their site contains accessibility 
features to accommodate the needs of users with disabilities. There is no charge to use 
this symbol in electronic or printed form. According to NCAM, more than 150 Web sites 
are displaying the symbol. NCAM guidelines should be followed when using the Web 
Access Symbol, text tag, and description to indicate that efforts were made to 
accommodate disabled viewers. (CPBAVGBH National Center for Accessible Media 
(NCAM): ncam.wgbh.org ) 





Eigure 1. Internet Accessibility. 



Eigure 1. Web Access 
Svmbol. 
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There is no way to guarantee that a site displaying the symbol will be 100% accessible or was even 
designed following the guidelines. However, in the spirit of the Internet, it is up to Web surfers to let the webmasters 
know when a site is or is not accessible and to offer suggestions for greater accessibility. 

Another trend, “Rich media”, refers to elements on a Web page (or in a separate player) that exhibit dynamic 
motion over time or in response to user interaction. Rich media players typically have menu items that allow captions 
and other accessibility features to be turned on or off. For more information about improving accessibility to Web 
pages on the Internet, visit the following sites: 

• BOBBY - Named for the British police officers, this site hosts a Web-based program that 
identifies accessibility problems on Web pages and teaches designers how to make sites 
more accessible. It also illustrates how Web pages will look via different browsers. 
http://www.cast.org/bobby/ 

• Web Accessibility Initiative - Sponsors information and 

resources. Take a look at some of the items (e.g., making \ t i 
a Website accessible, list of checkpoints for Web WW 

accessibility; quick tips for making Websites accessible) 

in the “Easy Introductions” section. http://www.w3.org/WAI/ 

• Designing More Usable Websites - Provides comprehensive 
links to references, tools, and numerous resources for 
increasing the accessibility of Websites. 
http://trace.wisc.edu/world/Web/ 

• Accessibility Tools for Adobe PDF Documents - Supports free tools that allow vision 
impaired users to read documents in Adobe PDF format, http://access.adobe.com/ 

• CPB/WGBH National Center for Accessible Media (NCAM) - A 
research and development facility that strives to make media 
accessible to underserved populations such as disabled persons, 
minority-language users, and people with low literacy skills, 
http : //nc am . wgbh . org/ 
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A Scenario Strategy to Promote Design Task Reflection 

Neal Shambaugh 

West Virginia University 

Abstract 

Scenarios provide a teaching strategy to promote reflection on design tasks in an instructional design (ID) course. 
Different approaches for generating and analyzing scenarios are described. A course summary provides a context 
for the use of scenarios. Data analyzed include student-developed scenario descriptions submitted pre- and post 
needs assessment and written student perceptions of the scenario activity. Initial scenarios produced improved 
clarity as to instructional problem and identification of project goals prior to needs assessment. Revised scenarios 
included narrative descriptions of successful implementations and new teaching options. Examples of student- 
developed scenarios are provided. 

Background 



Our conscious need to make decisions frequently takes precedence over thinking about the implications of 
these decisions. Such a need may be due to a human propensity to proceed to a solution in light of existing 
information (Simon, 1996) and the desire to act to decrease uncertainty. Disciplines, such as medicine, law, or 
engineering, may have formalized knowledge bases and guidelines for taking action (i.e., method) to address 
problems. Human problems do not lend themselves so easily, but are not out of reach of assistance. Instructional 
problems, those needs in which educational interventions are warranted, are forms of human problems that are 
complex. Schon (1983) characterized the differences in terms of terrain. As humans we prefer to keep to the high 
ground, which is hard and stable, rich with knowledge and tested principles. Instructional problems lie in the muck of 
the swamp, those “lowlands.... incapable of technical solution (p. 42). Design disciplines often work in “domains for 
which certain critical scientific and engineering knowledge is missing’’ (Carroll, 2000, p. 63). Instmctional design 
provides a systematic structure to think about and develop responses to instructional problems. Instructional 
designers face the same murky lowlands as other designers, but their role is crucial, as designers of all persuasions 
desire to take existing situations and improve upon them (Simon, 1996). The human world needs design. 

In ongoing research on teaching instructional design (Shambaugh & Magliaro, 2001), novice designers tend 
to propose a solution to an instructional problem without a clear understanding of the problem. However, reflective 
critique of these decisions is frequently removed from design activity. For example, in classroom settings in which 
instructional design is being taught, students typically hand in design work and make revisions based on instructor 
feedback. This traditional form of learning activity, however, distances students from thinking about responsive 
design decisions, those that directly impact learners. Development work, involving teams of designers, users, and 
developers, typically involves periodic or benchmark reviews, but these may be limited to specific technical features 
of the work without appraising the overall potential of the design to address learner needs. 

The use of scenarios is proposed to address this de-coupling of reflection from design. Scenarios record an 
envisioning of an approach or tentative solution and record our human tendency to move straight to a solution. 
Designer-constructed scenarios, tapping existing experience and information, provide condensed descriptions of 
proposed solutions to instructional problems. Scenarios, according to Carroll (2000), are concrete in that they 
propose a solution, but are also flexible as they are incomplete and easily revised. Scenario descriptions communicate 
a vision of intent in the context of what the designer knows. Analysis of the scenario, or claims analysis as labeled by 
Carroll, provides a means to evaluate the solution and improve upon it. 

The scenario description provides a narrative intent, while an analysis provides an inquiry into why the 
scenarios occur. Scenario analysis provides a general and explicit representation of the causal relationships that 
constitute the scenarios (Carroll, 2000, p. 272) so that propositions or hypotheses about usability can be made. 
Analysis of scenario descriptions may involve several approaches, as outlined by Carroll (2000), in an effort to see 
which approach works best in different circumstances. These analysis methods can include (a) cause and effect. 
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summarizing subject/verbs in the text and evaluating the causal relationships; (b) participatory analysis, in which 
student groups generate features and consequences in a scenario; (c) systematic questioning by the instructor (in a 
personal conference or on paper drafts) of each event, act, or intent in a description; (d) questioning stages of action 
at each phase ID (student drafts); (e) reuse of prior analyses, such as a match of learning types with intervention 
types; and (f) theory -based, involving designs based on a learning theory. 

Used in an instructional design (ID) course, scenario construction becomes a learning task within different 
phases of ID. In this way, ID novices build on what they know and reflect on the design as they design. For example, 
on the basis of a scenario description of a proposed educational intervention during needs assessment, novice 
designers will be asked questions and prompted to learn more about the instructional problem. From what is learned 
from needs assessment, students revise their initial scenario description, which is the basis for subsequent 
questioning, reflection, and design moves. Thus, scenarios provide a tool to envision a learning situation and what 
might occur in these envisioned settings. Scenarios provide an application of the situated perspective, that “. . . 
learning [is] a continuous, life-long process resulting from acting in situations” (Brown, Collins & Duguid, 1989, p. 
33). Thus, the research question in this study was “In what ways do scenarios assist novice designers in thinking 
through their instructional design decisions?” 

Methodology 



Course Design 

During a 15-week semester (Fall, 2002), 20 students in a master’s level instructional design course identified 
an instructional problem of their choice and designed a response to this educational need. Shambaugh and Magliaro 
(1997) was used as the text, which provided the instructional sequence for the course. The first two phases in the ID 
process. Learning Beliefs and Design Tools (principally ID models), were used to establish the context for the 
traditional ID process. Design-A -Lesson and Learning Principles tasks helped students to reflect on how they 
currently plan instruction and their views on learning. Students drafted a Mission Statement, which was used to 
assess how students' learning beliefs were being applied in their projects. Students also sketched a visual of their 
own ID model and provided an explanatory narrative. 

Students identified an instructional need and recorded their initial understanding of the problem through an 
Intent Statement. Prior to a needs assessment, students were asked to write a scenario envisioning a successful 
implementation of the project. The subsequent Needs Assessment phase structured students' research into the 
instructional problem and options for addressing it. A personal conference provided individual assistance with their 
needs assessment strategy in terms of what to study, who to talk with, what references to consult, and how to 
summarize their findings. Based on Needs Assessment, students identified goals for their project and revised their 
original design intent, including a revised scenario. Students were then introduced to design phases, which included 
Instructional Sequence, Assessment, Instructional Framework, Instructional Media/Technology, and a Prototype 
instruction (e.g., unit, web pages, tutorial, workshop activities). Instruction helped to raise students’ awareness to 
the purposes of assessment and appropriate assessment methods. During the Instructional Frameworks phase in 
which teaching options were specified, students demonstrated a teaching strategy in their Prototype Lesson. 
Instructional Media/Technology was addressed throughout the course beginning with Needs Assessment. Students 
were prompted to be open to media and technology possibilities and to make a case for how their choices supported 
their goals. A second personal conference addressed Program Evaluation and project issues. During the final week of 
the course students revised their personal ID models and submitted a written self-evaluation of the course and their 
learning. 

Data Sources And Analysis 

Three data sources comprised this study and included initial scenarios, revised scenarios, and student 
perceptions of the scenario task. Prior to conducting a needs assessment for their ID project, students developed a 
scenario description in a class session (week four) using work groups composed of individuals with similar projects. 
This initial group writing and sharing provided students with a draft from which to individually revise and submit the 
following week. This initial scenario description submission consisted of three sections: “Vision statement” 
describing a succes sful implementation of their design, a “Reality” statement qualifying the constraints on the vision. 
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and “Next Steps’’ or new design decisions. A table was constructed, which recorded a summary of what the 20 
students wrote for vision, reality, and next steps (design decisions). 

During needs assessment, students conducted research on the content to be taught, the range of learners, 
and the reality of the context (i.e., context analysis). Outcomes of the needs assessment included a revised Project 
Intent document, which summarized the major features of the proposed educational intervention, including project 
goals and a revised scenario. Based on what students wrote in the revised scenario description, a table summarized 
each student’s instructional problem and overall design strategy for subsequent development in the design phases 
of the course. Using this table, a cross-students analysis was then conducted to provide a general sense of what 
students were writing in the initial (pre-needs assessment) and revis ed scenarios (post-needs assessment). 

One of the difficulties of studying teaching interventions is the inherent conflict between teaching and 
research. As all students were afforded the opportunity to write a scenario description, it is difficult to know to what 
extent the intervention can be attributed with improving design decisions. The baseline for comparison is the 
developmental research studies conducted on this course from 1994 to the present and summarized in Shambaugh 
and Magliaro (2001). During this collaborative work, we observed student difficulties with clearly expressing the 
instructional problem while they designed. Students have mixed performances in their needs assessment activity, 
some rather resistant to the complexity of the activity as an academic task, while others had difficulty in taking the 
data derived from the needs assessment and incorporating their findings into to their subsequent design decisions. 
The scenario strategy was tried to prompt student thinking and reflection on design tasks during design activity. 
Although scenario descriptions were implemented prior to the design phases of the course, we have found that 
students respond to the ID process on the basis of their initial design intent. 

To partially address the value of scenario activity and its benefit for novice designers, a third data source 
asked students to record their perceptions of the scenario activity. During an in-class activity (week 5) students were 
asked to respond to two questions: “In what ways did the scenario activity and the re-write help your thinking? How 
can we make this a better activity?’’ Sixteen students were present in class and responded to these questions. 

Results 



Initial Scenario Descriptions 

From the initial scenario activity, students wrote more clearly about what they wanted to accomplish with 
their project than in their first design activity, which asked them in week one of the course to identify an instructional 
need and how they would address this need. In the “Next Steps” component, students wrote about adjustments to 
their original ideas. For example, changing grade level from middle grades to fifth grade based on the person’s 
teaching experience and learner needs in that grade. Several students made critical decisions as to their learners. This 
was most common with students who were teachers. These teachers wrestled with this question: Were the learners 
students in their classrooms or teachers like themselves? Several students wrote about the tensions they felt in their 
Vision-Reality statements, such as helping their students to learn the content, balancing software skill learning with 
its use in learning other content, deciding “who are my learners?” and searching for different ways to teach. The 
forms of the scenario written by students generally involved a first person statement of “this is what I want to do.” 

Revised Scenario Descriptions 

The revised scenario description included shorter scenario descriptions. These revised scenarios, which 
were submitted after conducting a needs assessment, included details on new teaching or assessment approaches, 
such as the use of games for teaching multiplication, 3-D geometric worlds, and reflective journals (as opposed to a 
workbook). Design strategies included specific details on length of the intervention, target learners, and specific 
activities for the teacher and the student. The form for the scenarios evolved from the “this is what I want to do” to 
an increasing use of narrative to depict the implementation. These narratives consisted of several points -of-view, 
including Class Scenarios, in which the scenario described how teaching unfolded. In addition, some scenarios 
focused on a hypothetical Student Case or Teacher Case, and the specific use of innovative Learning Activities. Two 
student scenarios, labeled as Future Scenarios, focused on both short-term and long-term goals. 

Student Perceptions Of The Scenario Task 



326 




Student perceptions were organized by what they wrote about their individual thinking and group activity. 
Students reported that the scenario activity gave them an opportunity to visualize and re-evaluate their original ideas, 
to “edit the dream’’ as one student wrote. Several students wrote about the value of talking in class and then 
rewriting their notes to link ideas, throw out others, and articulate the words for the instructional problem. One 
student commented that the task “made me really push myself to identify resources and constraints,” while others 
commented that the scenario “helped me to think more deeply and to think about the activity differently.” 

The group activity where students paired up to discuss each other’s vision for their project reported the 
value of “bouncing ideas of another,” that the conversation “sharpened my own sense of what I am doing.” An 
interesting comment worth noting was that “I viewed the project from the outside and having to tell others helped me 
to think more about the details.” 

Student Scenario Examples 

The first student scenario example involved a substitute mathematics teacher who proposed to address the 
“dreaded word/story problems” students face in math courses. Her initial vision at week three in the course was to 
teach a complete unit in problem solving starting with simple logic activities. She observed that in elementary schools 
the “word problem problem” is aggravated by teachers who themselves are least competent in math. One of her Next 
Steps was to focus her ID project on fifth graders rather than middle school students because fifth grade students 
may have a teacher who does not excel in math skills. The revised scenario consisted of two parts; the first, a memory 
of an early substitute teaching experience while teaching fifth grade math. She was preparing to review a set of story 
problems at the end of a text chapter. The students reacted with “We don’t do story problems in this room” and 
“Even our teacher hates story problems.” Her memory recounted the use of a mystery game to allow students to use 
their curiosity to complete the story problems. The second part of the revised scenario included a narrative of 
teaching a math lesson using a story as the basis for problem solving. Subsequent teaching moves asked students to 
find out what they needed to solve the problem, encouraging the use of trial and error, discovery, and reviewing all 
problem solving strategies. Akin to Duckworth’s (1987) “virtue of not knowing,” this substitute teacher envisioned 
students with “time to plan, discover, fail, and succeed” with helpful tips and hints to guide the students to master 
problem solving. As a partial result of the scenario activity (and needs assessment), the student proposed a five-day 
logic unit. Growing Solutions, to help fifth grade students learn logical thinking skills. She proposed both individual 
and group activities and incorporating enough time into the schedule for student examples and review. Thus, her 
initial view of content as “solving word problems” expanded to include basic problem solving skills, logic, and 
overcoming a dislike for word problems. 

A second student scenario development involved a university library staff person who proposed to create 
improved training materials for student library assistants. In his vision statement, the student identified four 
outcomes that would eventually become labeled as explicit project goals following a needs assessment. These goals 
included understanding of basic library access services, standardizing training content, appreciation for patron 
needs, and training on tasks within relevant library contexts. The reality of the current situation was the use of paper 
forms and outlines and inconsistent training across trainees due mainly to low staffing. The student’s decision of 
Next Steps “only served to deepen my original intent to standardize instruction and to go beyond text and oral 
demonstration to include a more sensory experience” via an integrated package of self-paced multimedia tutorials. In 
the revised scenario a training sequence involving an imaginary trainee was described using self-paced 
PowerPont/multimedia modules, which served to standardize the training of skills. The student also envisioned that 
the modules would contain interactive features to provide feedback to the trainee and her individual learning style. 
The subsequent design proposal was entitled Bringing Learners and Library Skills Together and adopted the goals 
originally identified in the original scenario description. 

A third student scenario, written by a teacher educator, envisioned a module for her Language Arts 
Methods course. The purpose of the course is to teach children the writing process, structure of text, grammar and 
spelling, and speaking and listening skills. The module was envisioned to specifically addresses sentence structure 
and grammar and to somehow teach students in the context of writing rather than through the use of worksheets. She 
wanted to use Internet sites to show examples of children’s literature and examples of grammar. She wanted to create 
instruction that allowed time for pre-service students to try out ideas with each other. In her Reality statements, she 
identified lack of computers in her classroom, lack of her own personal organization, and an inadequate textbook 
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section on grammar. In Next Steps she realized she could take her students to the library, that she would have to 
develop her own knowledge base for her students, and change her teaching from worksheets to students engaged in 
finding grammar examples in children’s literature. In her revised scenario, this student used a student survey in her 
needs assessment to find out more from her pre -service students in terms of their attitudes toward grammar. She was 
surprised to find out that many wanted more knowledge of basic grammar rules and that their idea of grammar was 
grammar rules, not the use of grammar in writing. She proposed activities in the children’s section of the library where 
students worked in small groups to locate examples of grammar concepts. She imagined students helping each other 
to find samples for use in writing. Her proposed project Teaching Grammar in Context identified the instructional 
problem as a lack of instructional strategies for teaching of grammar and a lack of a knowledge base of grammar rules. 
Her project proposed that her pre -service teachers design lessons for real life situations rather than the use of “skill 
and drill” lessons. Her project goals included an understanding of how to teach grammar to elementary students; 
designing lessons appropriate forNCATE, NCTE, ACEI, INTASC, and state standards; changing an attitude of 
teachng “skill and drill” to teaching grammar in context; and incorporating technology into their lessons. 

Implications for Future ID Instmction 

As this strategy is currently being implemented, completed ID projects were unavailable to help judge the 
overall usefulness of scenarios. Based on the available data, the initial scenario description (Vision, Reality, Next 
Steps) allowed students to address the mental tension between what they would like to accomplish and what they 
realized could be implemented given their known constraints. In their comments, students wrote that the task forced 
them to think through their plans. Based on the analysis of the initial scenarios, students wrote more clearly about 
the instructional problem than earlier attempts and identified specific intent that became goals for their ID project. As 
students generated goals prior to a needs assessment, these initial goals could be used to structure their subsequent 
analysis activity. An interesting question is to what extent should needs assessment be modified or even eliminated? 
This option may be worth considering in actual development work in which team members employ scenarios in a 
continual cycle of design-reflect-reframe. However, the value of needs assessment as a tool in an introductory ID 
course to learn more about content, learners, and context still appears useful. 

One student asked that scenarios be implemented earlier than the fifth week of the course. One possibility 
for future ID instruction would be to forefront a scenario in class sessions to help students experience each phase of 
instructional design. A multimedia counterpart to the scenario might be considered for use out-of-class or online use 
to simulate collaborative design-reflect-reframe activity. Analyzing the revised scenarios provided a set of categories 
(the predominant form being a Class Scenario) to help future ID students address this scenario task in individual 
ways, as it was a new task to them and additional assistance is needed to help them. Suggesting a Class Scenario 
night not be effective for all students, as their projects differ in terms of learners and intervention types (e.g., units, 
web sites, workshops). 

Another adjustment for future use would be to increase the use of scenarios in design teams. This might be 
effective if a class-based instructional problem or case study is used for ID instruction. Scenarios as design artifacts 
provide for designer reflection and dialogue (i.e., participatory design) through ongoing cycles of designing, 
reflecting, and reframing of the design problem based on growing understanding of the instructional problem and 
awareness of options and realities. One implication of scenarios is that some aspects of design decisions will be made 
explicit, while other aspects remain implicit and one may be unsure as to which occurs in these descriptions. 
Acknowledging that scenarios are limited by their incompleteness, they may still provide rich descriptions of the 
complexity of design work and how the designer views the problem and subsequent design response. Students 
seemed to value the activity. One reported that “when we start sharing, our thinking becomes clearer and additional 
ideas surface.” Another student commented that the task forced her to admit that she needed to think more about the 
problem and during the re-write identified new ideas or places to search. Such statements suggest that the scenario 
merits consideration as a learning activity, particularly when connected to needs assessment. 

A short-term goal of this research is to study how the scenario strategy could be used throughout the ID 
process, similar to Carroll’s scenario -system development (2000), which still uses a requirements -design-evaluation 
cycle of activity, similar to the traditional ADDIE (i.e., analysis, design, development, implementation, evaluation) 
model. Erom this initial study, students generated statements, which became explicit goals for their instructional 
design project. These orienting goals could “seed” an iterative cycle of scenarios in which human needs. 
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opportunities, and evaluation keeps design reflection of individuals or teams tightly involved within design activity. 
A subsequent step would be to use this scenario -based ID process to develop an instructional innovation. The 
information flow of this process, consisting of scenario descriptions and dialogue regarding these descriptions, 
provides a means to document the development. 

The above activities contribute then to a long-term goal of this research, which seeks to formalize what is 
learned from design efforts for use in future design activities. “If we can and do generate lots of candidate scenarios 
and claims, we will have lots of guidance in specifying, prototyping, and evaluating designs” (Carroll, 2000, p. 283). 
Such formalization of "lessons learned” might take the form of design principles generalized across some dimensions, 
such as subject-specific content, learning outcomes, or activity type, among others. Formalizations might also be 
categorized in terms of frameworks, such as taxonomies, developmental stages or issues, or implementation strategies 
(e.g., technology integration in schools). Ultimately, such formalizations would contribute to the development of 
explicit, archivable, and retrievable knowledge bases for particular disciplines. 
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Abstract 

In an effort to give Physician Assistant students practice in diagnosing and treating medical conditions, the Center 
of Excellence for Remote and Medically Under-Served Areas ( CERMUSA ) at Saint Francis University in Loretto, 
Pennsylvania, has developed web-based case studies for integration into the Clinical Pharmacology course that is 
taught at a distance. Designing the web-based case study learning environment required skillful utilization of 
multiple technologies. This paper focuses on the development methods and technology utilized in creating this 
unique learning experience. 

Background and Rationale 

Physician assistants are health care providers that are licensed to practice medicine with the supervision of 
a physician. Like a physician, physician assistants conduct physical exams, diagnose and treat illnesses, order and 
interpret tests, counsel on preventive health care, assist in surgery, and in most states they can write prescriptions. 
Physician assistants receive a broad education in medicine that is designed to complement physician training. 

One of the hardest tasks of a physician assistant is correctly diagnosing and treating medical conditions. In 
clinical practice, physician assistants must gather all of the inhibiting and limiting factors to determine the cause of 
the problem and treat the problem. In some PA programs, the students learn the principles of pharmacology in one 
course and they learn direct application of the pharmaceutical principles in other courses. Since these skills are 
emphasized in two different courses, the students have exhibited a difficult time integrating the skills in preparation 
for their clinical rotations. 

In an effort to combine both the cognitive and clinical aspects of this learning situation, the Center of 
Excellence for Remote and Medically Under-Served Areas (CERMUSA) at Saint Erancis University has developed 
web-based case studies. These case studies are integrated into the Clinical Pharmacology course that is delivered to 
physician assistant students at a distance. These case studies have been developed to give the students a realistic 
opportunity to apply their skills of diagnosing and treating patient medical problems in a secure environment where 
wrong judgments will not have a life threatening or negative effect on the patient. To begin the integration of the 
web-based case studies into the Clinical Pharmacology course, four case studies were developed to preceed each of 
the four examinations given during the spring 2002 semester. 

Outlining the Case Study Environment 

Of course with all instructional design situations, the first thing to determine is the goals or objectives of the 
learning experience. Eor the case study, the design team sat down and discussed what the goals of this unique 
learning experience were to be. The team decided on the following goals: 

Give physician assistant students the opportunity to diagnose and treat medical conditions. 

Simulate an actual doctor visit. 

Track the actions that PA students follow during a real office visit. 

Be accessible anywhere, at anytime, on any computer with Internet access. 

Since the case study environment was to reflect a real doctor experience, the case was divided into the 
steps or tasks that a physician assistant or Doctor would follow when diagnosing and treating patients. The case 
study was divided into four components: 

Welcome - This area is designed to introduce the student to the learning environment. The students are 
informed of the task they must complete and how to navigate throughout the learning environment. Also in this 
section, the student is introduced to the patient. With every doctor’s visit, the nurse comes into the room to take the 
patient’s vitals and chief complaint. This information is passed on to the doctor or physician assistant before they 
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enter the examination room to meet the patient. To ensure that the learning experience was simulating an actual 
doctor visit, the vitals and chief complaint of the patient is presented to the student upon entry into the case study. 

Question the patient - An important part of any doctor visit is the dialog that occurs between the patient 
and the examiner. It is through this dialogue that the medical professional obtains the current symptoms of the 
patient, the history of the illness, the patient’s past medical history, and the patient’s background. Depending on the 
illness, the doctor or physician assistant may ask a lot of questions or may ask just a few questions narrowing in on 
the current disease state. In this area of the case study, the students can choose the questions that he/she wants to 
ask the patient. To simulate this vital patient/medical provider dialogue, the students see a video of the physician 
assistant asking the question and the patient’s response to the question. 

Examine the patient - In order to diagnose the patient, a physician assistant will perform various physical 
examinations on the patient. In this area of the case study, the students can choose the examinations that they wis h 
to perform or order. The students see a video of the examination being performed as well as the results of that exam. 

Diagnose and Treat- To end a doctor’s visit, the patient is told the diagnosis and is given a course of 
treatment. In this area of the case study environment, the student must determine the diagnosis and prescribe a 
course of treatment. Of course, the physician assistant or doctor always explains the diagnosis and treatment to you 
along with some patient counseling issues. In the case study environment, the students submit this dialogue in a 
text box. When completed, the diagnosis and treatment form is submitted to the instructor for grading. The student 
will receive their grade and feedback from the instructor through the course e-mail. 

Developing the Cases 

Once the case study environment was outlined, it was time to begin developing the cases. In the 
development process, the design team had to answer two major questions: 

How can we produce realistic videos to incorporate into the case study that showcase the events of a 
doctor visit? 

How can we develop a web-based interface that will accomplish the goals of the learning environment and 
remain user-friendly given our intended audience? 

Many different approaches, designs, and techniques were tried in our initial four case studies. 

Developing realistic videos 



Developing the script 

The first step in developing the videos to be used in the case study was scripting the patient/physician 
dialogue for each case. One of the objectives of the learning experience was to simulate a real doctor’s visit. 
Therefore, it was important that the characters appeared real but it was even more important that the medical content 
presented in the case was correct. Several different approaches to writing the script were taken to determine which 
method produced the most realistic patient visit. 

The first method of developing the script was for the instructional designer and pharmacology instructor to 
research the medical condition in various medical textbooks, journals, and Internet resources (ie. Yahoo Health, 
MDConsult). After researching the signs, symptoms, and causes of the medical condition, a preliminary script was 
written. This script was then sent to a medical doctor who checked the script for medical accuracy and provided 
several revisions. Once revised, the script was given to the actors. During the taping of the scenario, the actors may 
“ad lib’’ some. The changes to the script were noted and sent back to the medical doctor to ensure that the case 
remained medically sound. This method worked well as a way to ensure medical accuracy, however, the actors 
seemed rehearsed and the script did not reflect the personality of the medical examiner. 

Figure 1 

First Method of Writing the Script 
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Research 




A second method of developing the script was to tape the case completely unscripted. A patient who had 
or is currently having the desired medical condition was found. The patient was brought into the production studio 
and the physician assistant performed the examination with the patient completely unscripted. For production 
purposes, the examination was taped multiple times at differing angles. The dialogue between the actors changed 
each time the examination was performed. Following the taping, the audio was transcribed and then collated to 
develop a script. The script was then sent to the medical doctor for medical accuracy and to ensure that the case was 
medically sound. This method produced a very realistic exchange between the patient and the physician assistant, 
however, the post production work of transcribing and collating the script was very time consuming, and therefore, 
would not be cost effective when producing multiple cases. 

Figure 2 

Second Method of Writing the Script 




Currently being tested is a third method of writing the scripts as we continue with our development of case 
studies and work toward the development goal of one case per medical topic covered in the course. The topics to be 
developed were divided between two members of the case study design team, a medical doctor, and a physician 
assistant. Each of these medical professionals is writing the scripts for the case studies that they will be the on 
camera talent for. Once they have developed the script, the two medical professionals meet to share the scripts they 
have written, as well as check each other for medical accuracy. This method is still in it’s preliminary stages but has 
so far made the doctor and physician assistant very realistic as the dialogue and examination truly reflect their 
individual personalities. 



Figure 3 

Third Method of Writing the Script 
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Taping the video 

Once the case study was scripted and actors were determined, we were ready to shoot the video. Since the 
pharmacology case studies were to have taken place in a doctor’s office, CERMUSA had two choices on how to 
show this: 

Take our cameras, lights, mics, and cables out into the field at a real doctor’s office and shoot the video 

there. 

Create a doctor’s office virtually by using chroma-key technology. Chroma Key, or green-screen 
technology has been used in the television industry for 40 years. It consists of a video processor that substitutes an 
image onto a color in the background, usually a green or blue colored field. The technology is most commonly used 
in television weather forecasts. CERMUSA’ s Distance Learning Prototype Laboratory has its own chroma key 
background in its studio. 

Eigure 4 

Image of Chroma Key in Action 




Since the first option wasn’t cost effective, we went with the latter. In order to create the background, 
CERMUSA staff members went to a doctor’s office and took pictures of the examination room in what would 
approximate a patient’s view and a doctor’s view. The images were then imported into Adobe Photoshop where a 
slight blur was applied to give the impression of depth of field for the production. 

During the taping of the video, it was decided to shoot the video in individual shot takes. First take would 
be the two-shot, the doctor and patient in the examination room. Then, individual close-up shots of the doctor and 
patient were taken. This way, we would be able to have more shots to choose from in the editing, thereby making the 
resultant video visually appealing. 

During the video production/editing, an option presented itself regarding the chroma key: Do we put the 
image in during the shoot, much like a TV weather broadcast, or do we add it in afterwards in our non-linear editor 
(NLE). 

At first, we tried adding the chroma key image in our NLE. This process allowed us to insert any picture that 
we wanted into the video, increasing our flexibility. However, this process was time consuming, as the background 
image had to be rendered into the background color. For example, a ten second clip would render in approximately 
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three minutes. When one is dealing with a two or three minute clip, this amount of time can be prohibitive to an 
efficient production time. Therefore, even with the benefit of slightly more flexibility in image selection, the time 
involved made this situation less than ideal. 

The second option, adding the image during the original taping, turned out to be the better option. With 
some planning, CERMUSA could add any image it wanted into the background. It also freed up the time it took to 
render the image into the background. This option was found to be the best option in terms of getting the clips 
edited and out to the students in a timely manner. 

Editing the video clips 

Shooting the video gets you the images that you need; editing gets you the story that you want to tell. In 
terms of the medical content, the editing didn’t make much of a difference as the content was contained in the 
dialogue. However, the vis ual images and the way they were arranged made the content more appealing. 

Using a DPS Velocity 7.6 Non Linear Editor (NLE), CERMUSA was able to make the individual clips, make 
any changes to those clips that would come up, and encode the clips into the streaming media of choice. The 
individual clip production was similar to cutting and pasting in a word processor. One would be able to paste a 
portion of video, trim the time to get the desired length, and remove a piece of that video if it was necessary. This 
enabled the video editor to be able to add in other elements, such as still pictures, for example, adding a picture of a 
fungal skin infection. Einally, the NLE had the programs to encode the resultant video into the streaming media. 

The look of the pieces was determined once again by the content. The video clips were edited so that if the 
doctor or patient was saying something important, or distracting, as the case may be, typically the single close up 
shot was chosen in order to draw the attention of the audience. Eor the general, “How are you doing?’’ type 
questions, as well as anything that would require a wider shot, such as an examination, the wider two-shot was 
selected. 



Developing the web- based interface 

Perhaps the most difficult aspect of developing the case study learning environment was the development 
of the web-based interface that would be used. This interface was the engine that the students would interact with. 
Through the web-based interface, the students would receive the medical content, enhance their clinical skills, and 
practice diagnosing and treating patients. Since the Clinical Pharmacology course is taught at a distance, it was 
important that the web-based interface be designed to fit the needs and technical skills of our intended audience. 

To begin the development of the web-based interface, storyboards were created. These paper based 
drawings of the web-based interface outlined how the material would be presented to the students. The location of 
the navigational buttons and content material were essential consideration in this design. The design team knew it 
was important to be consistent with the layout, placing similar items in the same location throughout the web 
environment. 

Once the look of the interface was determined and outlined in the storyboards, the design team had to 
determine the technology that would be utilized to create this unique learning environment. 

Delivery of the video 

One of the major hurdles in developing the web-based interface was determining how we would deliver the 
video clips to the students. These video clips are important to the quality of the case study. They are the added 
flavor that highlights the skills and techniques of a practicing medical professional. 

The first method of delivery that was considered was to distribute the clips on a single CD media. An initial 
case study developed and implemented in Phase 1 of the project (2000 to 2001) used this delivery method. This 
solution posed many obstacles in a long run. One problem was maintaining the videos that were distributed to the 
students and keeping the medical content presented current and accurate. Medicine is a field that is constantly 
changing, as research is done, treatments for medical problems change along with the signs and ways of diagnosing 
the condition. Understanding the fact that at times we may need to change the clips to reflect the current medical 
practice, the idea of issuing the video clips on CD to the students became less acceptable. The final problem with this 
delivery method was that using CD-Rom media requires extra lines of codes and the use of cookies to determine the 
right drive for the CD drive. 
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After looking at problems associated with CD media, the idea of streaming video became the best choice of 
video delivery. Unlike video downloads, where one can easily spend half an hour downloading a video file onto your 
hard disk, in streaming video the video content is downloaded in small chunks so that it will start playing almost as 
soon as you start downloading. While the video is playing, the computer is downloading the next chunk. Streaming 
video technology thus gives virtually anyone the capability of providing an Internet based, on demand video 
broadcast. Currently the only real drawback to the technology is limitations on communications bandwidth, the 
amount of data you can send over a dial-up connection in a given period. This means that the displayed image must 
be delivered at a fairly low resolution. 

From the video maker's point of view, what is great about streaming video is that it does not require any 
physical distribution, and therefore allows one to distribute to a global market, without the problems of postal 
restrictions and media format differences. It is also a technology which is very cheap and is easily used by someone 
wishing to distribute a single homemade video, as well as a big company marketing dozens of films. 

For the first run of streaming video in the case study, we decided to use Real Media software for the 
delivery of the video. However, a major issue that we encountered with this software was that it, along with other 
video software, required the player software to be installed on the client computer. This was a major concern for us 
and it required the users to install the player on their computers to view the video clips. As the students were 
accessing the case studies in computer labs at their institution, administrator rights set up on the computer lab PCs 
prevented the ability to download the necessary software on the local comp uter. 

A solution to this problem was provided with Microsoft’s Windows Media Player software. Most students 
have the Windows Media Player already installed on their Windows Operating System. Since it is desired to deliver 
the content as smooth and trouble free as possible to the user, utilizing this player was ideal. Another bonus to this 
software is that the server version of the media streaming was included with the Microsoft Windows 2000 Server. 

This reduced the cost of purchasing and upgrading the server software. 

Even after using the streaming server, the major problem became the delivering of the video content to the 
remote sites. The use of firewalls and Packeteer® restriction software (an application traffic and bandwidth 
management system that enables network administrators to keep critical traffic moving at an appropriate pace 
through bandwidth bottlenecks and prevents any single type of traffic from monopolizing the link) by the network 
administrators created a new challenge for us, as well as pushing the video through high network traffic utilization. 
CERMUSA’s Informational Technology staff came up with several possible solutions to this problem. 

The first possible solution was to provide cache boxes to the remote sites. The idea was to setup special 
devices inside the institutions network and push videos from our main streaming server to the cache box. The clips 
would reside inside their network in a cached area. This would speed up the delivery process drastically. However, 
the price of mentioned devices was extremely high. 



Eigure 5 

Cache Box Solution 




A second, and probably the best option was very much the same as the first but cheaper in price. Using a 
normal PC with increased memory and hard drives, we developed our own cache box that would host the video clips. 
Using specific codes in ASP, we would route the users to connect to this server to view the videos if they were using 
any computer within that institution’s network. At this time, we have installed a server at one of the remote 
institutions. Although we are still in the preliminary stages of testing this technique, the data thus far shows that the 
solution is working wonderfully. 



Eigure 6 
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Mirror Server Solution 




Tracking the students 

One of the goals of the learning environment was to track the path the PA students follow during a real 
office visit. This tracking allows us to understand the steps that they are taking in the process of diagnosing and 
treating the patient. The biggest challenge we had to face in order to track the activities was to decide what 
technique was the most common, and less troublesome, to employ. 

One option to track the students’ action was to use cookies. The main purpose of cookies is to identify 
users and possibly prepare customized Web pages for them. When you enter a Web site using cookies, you may be 
asked to fill out a form providing such information as your name and interests. This information is packaged into a 
cookie and sent to your Web browser which stores it for later use. The next time you go to the same Web site, your 
browser will send the cookie to the Web server. The server can use this information to present you with custom Web 
pages. So, for example, instead of seeing just a generic welcome page, you might see a welcome page with your name 
on it. 

But what are the downfalls of using cookies? The main problem is that many people disable their cookies on 
their favorite browser. By disabling the cookies in your browser, or in some cases installing software to block and 
remove them, it is mainly impossible to accomplish the task of tracking the student’s activities. 

For the case studies, we determined that we would use Active Server Pages, or ASP for development and 
tracking purposes. ASP, provide a means for us to activate the case study web interface with dynamic live database- 
driven content. All these codes are server side activated. This means that Web server executes and delivers the 
content to the client, unlike the java scripts, so that they are not client side dependant. The major benefit to using 
ASP for development is that the end result is raw HTML so that there are no concerns if the visitor to the site is 
compatible with the language or tools we are using. Another benefit to server side Web development is that we can 
utilize components on our server without requiring the visitor to install any particular program. Almost 97% of the 
Case study web pages are ASP based. 

From the time that a student logs onto the site, his/her presence and activities are tracked and logged into a 
database with the utilization of session variables. A session, in ASP terms, is a single visitor traversing an ASP 
application. The session starts when the visitor browses to an ASP application or goes to an HTML page that has 
code that directs to an ASP. The session persists until the visitor has left the site. Visitors to the site can have their 
session terminated implicitly or explicitly. The session ends implicitly when they haven’t requested one of the pages 
for a predetermined number of minutes, in our case 20 minutes, or the session ends explicitly when there is a code or 
procedure to terminate the session variable. 

While the visitor travels through our site, we can use the session object to store information about the 
visitor and to control certain aspects of the visitor’s session. The session object includes collection, properties, and 
a method that you can code to provide this functionality. These session objects can be passed onto other pages and 
can be used by other pages for as long as the visitor stays on that web site. Once the visitor closes the web site, the 
session variable(s) will be destroyed. Unlike cookies, that are saved on the visitors hard drive, the session objects are 
client independent. An example using session objects can be seen below. 
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An example using Session objects: 



// Declaring the Session objects 
<% 

Session (“UserName”) = “John Smith” 
Session (“UserlD”) = 10 
Session (“City”) = Pittsburgh 
%> 



// Using the Sessbn objects 



<% 

If Session(“City”) = “Pittsburgh” Then 
Session (“City”) = “Pittsburgh, PA” 

End if 

Response.Write “Hello: ” & Session(“UserName”) & “.<P>” 

Response. Write “ Your account number is:” & Session(“UserID”) & “.<P>” 
Response.Write “How are things in “ & Session(“City”) & “?<P>” 

%> 



In our case study project one of the session variables that is used by all pages is the logon ID. This session 
variable is a unique variable that allows the pages to display or track only that specific user’s activities. An example 
is what video clips has the student watched so far? One of the features in the case studies that we use session 
variables for is the patient diagnosis page. This page is designed for the students, at the end of the visit, to make 
their diagnosis and submit for grading. Students can only complete this form once. As mentioned before, all student 
activities are securely tracked into a database. The user logon is one of the objects that are constantly monitored. By 
doing this, we can track if the student had taken the diagnosis exam before, and if so it is easily traceable by matching 
the session variable that holds his/her user logon with a field in the database that had saved the user logon in the 
previous visits to the site. 

Use of Java 

A lot of the functionality of the case study is driven by lava scripts and lava applets. 

• The first use of the java scripts is to detect the user’s browser. With the case studies, we wanted to be sure 
that the students were using Internet Explorer to access the case. The reason for this is that older versions 
of Netscape (4.X) can handle the ASP with many limitations. The biggest fault with Netscape is the lack of 
handling tables and layers. It does not handle the placements of the objects used on the pages. In order to 
correct those problems we had to write separate scripts to allow Netscape to display the page properly. 
Netscape also lacks the capability to display inline frames, which are used extensively on the case study 
web site. Inline frames are frames that do not follow the frame rules and regulations of normal HTML. They 
are designed to display another page within a page without creating a whole new HTML frame structure. 
These inline frames are used to display video clips and to show students their track of questions or 
examinations that they have been following on each page. 

• Another use of the java scripts is to maximize the browser to its full size on the user’s screen. This is 
accomplished by detecting the resolution of the screen and the code automatically adjusts the screen to that 
size accordingly. When the screen is maximized the menu and navigational options of the Internet Explorer 
browser window disappear. This is done for two reasons. 

1. To save as much space for displaying more data on the page as possible. 

2. To make sure that visitors to the site don’t get lost or make wrong clicks and leave the web site 
without finishing the learning activity. 
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• The java applets have been used to create slide down navigational menus for the visitors to make quicker 
choices. 

• Finally, java scripts have been used to track the student’s diagnosis and grade them according to their 
responses to the choices provided on the diagnosis page. All java scripts are hidden in separate files and 
are only referenced by a simple line of code. This insures that no one can see the codes and the results of 
the diagnosis. 

Conclusions 

As we went through the development of the initial four case studies, we tried the various methods and 
techniques outlined in this paper as we strived to develop the web-based case study learning environment. During 
the implementation of these four initial cases, we encountered several technical problems, other than the streaming 
video issues previously mentioned, that needed to be assessed and remedied. 

One problem that we encountered was the character size of the textbox in the diagnosis and treat section of 
the case study. Originally, this box was created for 1000 characters. As we implemented the various cases, the 
students had more to say to their patients and began writing their dialogue in Microsoft Word to copy and paste into 
the text box. This increased the number of characters needed in the textbox. The textbox was adjusted to allow 2000 
characters initially. Now as we continue developing more cases, the textbox has been readjusted to 8000 characters. 

With the case studies, a complete text based version was provided as well as the video version. The text 
only version of the case study provided a transcript of the dialogue of the video / audio version of the case study. 
The transcript appeared in the area of the screen where the video usually appeared. The text only version of the case 
study could be selected by the student when they clicked to begin the case. The design team included the text only 
version in the alpha case design for two reasons. These reasons were: 

1. To ensure that the web site met the requirement of the American’s With Disabilities Act (Section 508). 

2. To provide an alternative in the event of audio and video streaming failures. 

Over 60% of the students utilized the text version of the case study because of technical difficulties with 
delivering the video to the students. As mentioned before, the institutions’ network administrators were controlling 
their network traffic with racketeer®. At first, the racketeer® was blocking all streamed media from CERMUSA 
servers. After speaking with the network administrators, streamed media from the CERMUSA servers were placed as 
a high priority at both institutions. However, due to the other network traffic at both institutions, we still experienced 
long periods of waiting for the video streams to begin. Since the video version adds to the case by highlighting the 
procedures of a medical professional and showing the students howto correctly perform various physical 
examinations, finding a solution to the streaming video problem became essential for the future of the project along 
with adding more features to the videos to make them more essential for successful completion on the case study. 

A development goal is to develop a case study for each of the medical conditions covered in the Clinical 
Pharmacology course. This goal means that CERMUSA will be developing an additional 45 case studies. The 
development has been divided into two phases, half of the topics in one year and then developing the remaining 
cases the following year. 

As development continues, CERMUSA will continue to experiment with the video and web-based interface 
as technology and student utilization changes. 

This project is partially supported by the Center of Excellence for Remote and Medically Under-Served 
Areas ( CERMUSA ) at Saint Erancis University, Loretto, Pennsylvania, under the Office of Naval Research Grant 
Number N0001401 11002. 
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Constructing Learners in 3D: An investigation of Design Affordances and 
Constraints of Active Worlds’ Educational Universe 

Michele D. Dickey 
Miami University 

During the past decade, a proliferation of new and emerging technologies have been designed and adapted 
as environments for distance learning. Among the more interesting contenders of adapted technologies are three- 
dimensional (3D) virtual worlds. Three-dimensional virtual worlds can be roughly described as networked desktop 
virtual reality. While there are a variety of applications, typically most provide three important features: the illusion 
of 3D space, avatars that serve as visual representations of users, and an interactive chat environment for users to 
communicate with one another. Although most 3D virtual worlds have been designed primarily as social and gaming 
environments, one 3D virtual world application. Active Worlds, has delved into the educational arena by creating the 
3D virtual universe. Active Worlds Educational Universe, devoted solely to education. 

While 3D virtual worlds are a relatively new technology only recently being explored for use as distance 
learning environments, studies of text -based or chat virtual communities have provided compelling views of how 
technology influences the representation and social construction of users in a computer-mediated environment 
(Bmckman, 1997; Riner 1996; Turkle, 1995). Like text -based chat communities, 3D virtual worlds provide users with a 
sense of presence. Participants can engage in conversation, build, and interact within an environment. Unlike text- 
based chat environments, virtual worlds afford users a visual representation of place, space, and self that provides 
users with a sense of immersion and embodiment in the environment. While virtual worlds do provide a sense of 
embodiment, it is important to note that this embodiment may be encoded with values and beliefs not readily 
apparent to users. Research into race, ethnicity, culture, and gender in virtual communities argues that virtual 
communities are embedded with values and hidden assumptions that promote the interests of some users, while 
marginalizing other users (Bailey, 1996; Balsamo, 1994; Branwyn, 1994; Cherny, 1995; Dibbell, 1994; Milthorp, 1996; 
Morningstar and Farmer, 1994; Stone, 1995; Todd 1996; Turkle 1995). While new technologies offer more options for 
distance learning, it is important that researchers, educators, and practitioners be aware of how an application may 
construct a user/learner through the values and hidden assumptions encoded within an application (DeVaney, 1993). 

Purpose of Study 

The purpose of this study is to investigate how the design affordances and constraints of Active Worlds 
Educational Universe impact the construction of learners and the culture of an educational three-dimensional virtual 
world setting. Specifically this investigation will examine affordances and constraints that impact: (a) presence; (b) 
individual and collective representation; and (c) embodiment. The context of this study is Active Worlds Educational 
Universe, a three-dimensional virtual world universe created and supported by the developers of Active Worlds. 

The goal of this study is to reveal how values and assumptions embedded in the design of an educational application 
impact the presence, representation, and embodiment of learners. 

Theoretical Framework 

The underlying question that guides this investigation is how do educational computer programs construct 
their subjects? Ann DeVaney (1993) first posed this question in an article entitled, Reading Educational Computer 
Programs . According to DeVaney, questions such as this are important for addressing often neglected culture -bound 
issues of educational technology. In an era of diversity and inclusion, it is important the educators be vigilant in 
addressing ways in which educational technology predispose learners to certain actions which may valorize some 
learners and depreciate others. 

Methodology 

The methodological framework employed in this investigation is a liberal adaptation of Affordance Theory. 
Affordance Theory, as described by ecological psychologist, James Gibson (1977), expresses the relationship that 

339 




exists between an animal (perceiver) and the environment (perceived). The relevance of liberally adapting Affordance 
Theory for an analysis of a 3D virtual worlds setting is that 3D virtual worlds are simulated environments. The 
affordances and constraints of the application impact opportunities for how an environment may be perceived. 

Methods employed in this qualitative investigation include participatory observations (Adler and Adler, 
1994), informal interviews (Fontana and Frey, 1994) with learners, developers, and educators of Active Worlds 
Educational Universe, and interactions and observations with my students using Active Worlds Educational 
Universe. 

Setting 

The setting for this investigation is Active Worlds Educational Universe. Active Worlds first premiered in 
1995. The client-server application consists of the Active Worlds Universe with over 1000 individual worlds for 
users to interact with other users worldwide. In 1999, the owners of Active Worlds created the Active Worlds 
Educational Universe, a universe devoted solely to education initiatives. The Active Worlds Educational Universe 
affords user-extensible provisions and support for building new worlds and adding to existing worlds. World owners 
are free to define and customize their world in whichever way they choose by selecting objects from the AWEDU 
object library or by adding custom built objects. 

The AWEDU browser interface is comprised of four main scalable windows which include a 3D 
environment; a chat dialogue window; an integrated web browser; and a window for added navigational and 
communicational functions (see Eigure 1). The 3D environment is the primary setting for interaction. Learners 
represented as avatars, move and interact with each other and the environment by either moving their mouse or by 
using arrow keys on their keyboard. Directly beneath the 3D environment is the chat window. Communication is 
limited to text messages which are display both above the speaker’s avatar in the 3D environment and in the chat 
window below. 

Eigure 1. The Active Worlds Educational Universe browser. 
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Findings and Analysis: Presence/Attendance 

Initially this investigation examined how the learner was constructed by focusing on three categories: 
presence, representation, and embodiment. Initially presence was determined by looking at the affordances and 
constraints of hardware and software requirements, skills necessary to participate, and language accessibility. 
However, as this study progressed, the categories that are emerging are not as first believed, but rather 
representation and embodiment are all elements that construct the learners Presence. The category initially labeled as 
presence now seems more aptly described as attendance. 

Findings 

There are various affordances and constraints that impact a learner’s attendance in the world. Hardware 
requirements include: Pentium 200mhz or equivalent, 64MB RAM, Microsoft Windows (95- XP), and DirectX 3.0. 
Currently there is no MAC version; however. Active Worlds Educational Universe is accessible on a Mac running 
Virtual PC. While there are few skills required to participate within the Active Worlds Educational Universe, basic 
skills required would include the ability or access to someone who can download the application from a website and 
install it. 

Learners are constructed by language both in AWEDU and in the documentation. AWEDU is one of the 
few 3D virtual worlds to support many languages. While English is most often observed in some of the more 
commonly visited worlds, it is by no means the only language used. Depending upon the time of day and location, 
English may not be observed being used at all. The AWEDU browser supports the following languages: Spanish, 
Danish, English, Dutch, Erench, German, Einnish, Italian, Hungarian, Norwegian, Portuguese, and Swedish. Despite 
language support, some of the browser features remain in English (Eigure 2.). English is als o the primary language 
used for documentation. While there are provisions for multi-language support, clearly some knowledge of English 
would be an advantage for learners in this environment. 

While the AWEDU browser officially only supports a select list of languages, it is not uncommon to find 
many other languages being used in various worlds including both Japanese and Russian. Additionally, since many 
languages use the same alphanumeric system as is used in English, many more languages are used than are 
represented by the browser. 
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Figure 2. The Italian language version of the AWEDU browser. 




Analysis 

Learner attendance is constructed by way of the hardware, software, downloading skills, and language. 
Learners must have access to a Pentium computer and have Internet access. In addition learners must possess the 
skills to be able to download an application and load it on a machine. This means the learner must also have the 
language skills to read the instructions or have access to a translation. 

Learners’ construction is impacted by language accessibility both in AWEDU and by access to 
documentation. Although the AWEDU browser supports multiple languages, the languages supported reflect 
primarily Western, European languages. Equally important to note is that not all functions are translated. Learners 
still to some extent must have English language skills or have access to a translation. 

Findings and Analysis: Individual and Collective Representation 

In AWEDU learner representation is constructed in various ways. The identity of the learner is essential for 
recognition, while avatars serve as the visual representation of learners in the 3D environment. While these two 
elements comprise and impact the direct construction of the learner, the environment also plays a strong role by 
illustrating world views displayed in the setting. 

Eindings: Identity 

In real life, identity is often tied to some physical aspect of an individual. Appearance, facial features and 
voice are among the means by which we distinguish one another. In the AWEDU environment, these physical cues 
are not available. Learners are not recognized by voice, or by their appearance, but instead must rely on unique 
identities to identify one another. Within the Active Worlds Educational Universe registered users may select a 
unique identity. This identity /name may not be used by any other leaner. This provision allows learners to recognize 
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and contact each other. A learner’s unique name appears above his/her avatar the first time he/she uses the chat tool 
during a session. If the learner wishes to remain anonymous or lurk, not “speaking” will insure anonymity. 

With a unique identity, learners may have extended communication options such maintaining a contact list 
which allows them to identify when another learner on their list is currently active. The may also send and receive 
telegrams from learners currently inhabiting other worlds. Although learners have the option of changing their 
unique name, any name change will be reflected in other learner’s contact lists. 

Analysis: Identity 

A unique identity brings both privileges and accountability. It allows learners to establish both reliability 
and consistency in both the personal and social arena. Learners can build, develop contact lists, send and receive 
telegrams. This is important for establishing a degree of trust among learners. Along with the privileges of a unique 
identity comes a degree of accountability. Within a system where learners adopt alias identities a unique identity 
prevents learners from impersonating one another. In turn, this affords learners with some degree of self-governance 
among learners . The accountability established by a unique identity can also impose inverse limits on well meaning 
learners. Once a learner establishes a unique identity and other learners may add him/her to their contact list, any 
subsequent changes in his/her adopted identity will be reflected on the other learners’ contact list. Initially this may 
not seem to impose limitations on how the learners represents themselves; however, in an environment where 
learners are free to re-construct themselves or develop alternate personae this can prove constrictive. 

Findings: Avatars. 

The creators of AWEDU define ''avatar'' as "the visual representation of people who currently inhabit the 
virtual environment. In an AWEDU world, avatars serve not only as the visual representation of a learner, but also as 
the camera into a 3D environment. 

AWEDU provides a stable of avatars that world owners may select and make available to learners visiting a 
world setting. Learners are limited to selecting from avatars provided by world owners. Because of the limited range 
of avatars, many learners may use the same avatar within an environment. Consequently, learners must rely upon the 
unique identify to identify other learners. 

With few exceptions the AWEDU avatars represent young, fit. Western, Caucasians. There are more male 
avatars than female, and it is interesting to note that there is a greater range of body types, sizes, ages, and styles of 
male avatars than female. Within the selection of female avatars, there is little variety in size, weight, age, ethnicity, 
and race. Most of the female avatars represent young shapely women with either snugly fitting clothing or short 
dresses and high heels. There are also no avatars representing differently -abled persons. There are, however, a few 
alternatives for learners not wanting to be represented as humans or as a gender typically in the form of a bird or 
alien-type caricature. 

AWEDU allows world owners to create and import custom-made avatars; however, creating an AWEDU 
avatar is no small feat, nor one easily accessible to many learners . Creating an AWEDU avatar requires a basic 
knowledge of 3D concepts. It also requires that a learner be fairly proficient with creating Renderware (RWX) objects 
or in using 3D modeling software such as Caligari’s Truespace. 

Analysis: Avatars 

Eor most learners there is little opportunity for creating custom avatars. With the exception of world owners, 
learners are confined to using one of the prefabricated avatars provided by an individual world. Eor visual 
representation, learners must rely on avatars that may not reflect their personal values, culture, ethnicity, or physical 
bodies. The avatars are primarily idealized representations of young adults that clearly reflect Western body-image 
values. 

Eindings: Environment 

Initially it may seem incongruent to include environment as a component in learner representation. 

However, in AWEDU while various factors may limit or prevent learners from creating and using custom-made 
avatars, many worlds provide opportunities for building within worlds. AWEDU is a user-extensible system which 
allows learners to add to and build within a world setting. Depending upon provisions determined by the owner of a 
world, learners can claim property by placing ground covering on a section and build. Once a piece of land is staked 
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or covered by a learner, no other user may build on that land, or above and below it. To build or add to property 
claim, a learner merely chooses from a selection of pre -fabricated objects that exists in the object library and places 
the object in the desired location. The objects in the library range from a variety of building materials such as walls, 
doors, and windows to specific items such as televisions, glasses, and cars. There are also a full range of objects 
such as flowers, trees, grass, and bushes that allow users to create an outdoor setting as well. 

Analysis : Environment 

Throughout most of the worlds in the AWEDU, are elaborate buildings and roadways constructed by 
learners and developers. These environments sometimes reflect places that exist in the physical world, and 
sometimes border on fantastic. One question worth asking is why is there a need for buildings and roads? In life, 
buildings serve many purposes, but primarily they offer us shelter from the environment. In a virtual world setting, 
there is no impact from the environment. It doesn’t rain, snow, and the temperature has no impact upon our avatars. 
There is no real sun and the ambient lighting never changes. Why have buildings? Why have ground or gravity? 
The stock objects reflect and anticipate how developers perceive AWEDU will be used. While the provision of stock 
objects affords an assessable means of building in a world, it also constrains how one might envision a world (Heim, 
1998). 

The objects provided by AWEDU also reflect values of how nature is to be regarded. While there are a 
plethora of tress, flowers, bushes and grass to select, it is important to note that there are no weeds, the grass is well 
groomed, and the trees never shed leaves or require attendance. Nature is contained and controlled. Nature is 
modeled on a culture in which is mechanized and submissive (Merchant, 1980). The values perpetuated clearly reflect 
Western values of nature and the environment with no mutual impact on either the environment from the avatars or 
upon the avatars from the environment. In AWEDU, nature is contained and subservient, and learners are both in 
control and impervious from impact. Both the types of objects for constructing structures and the objects available 
for constructing nature reflect also reflect Western values. 

Findings and Analysis: Embodiment 

It may seem somewhat incongruent to speak of embodiment in a computer-mediated environment such as 
AWEDU. Although there are avatars serving as representatives of learners, nothing really impacts the avatar. It 
doesn’t get hurt, feel hungry, or become tired. Avatars do not feel gravity or in anyway impact the environment. 
Despite that lack of kinesthetic cues, surprisingly research reveals that users in 3D virtual world settings often adopt 
and maintain proximity customs from in real-world interactions (Jeffery & Mark, 1998). Embodiment in AWEDU is 
constructed by way of animated avatar gestures, actions, and emotions; learner point-of-view and navigation; and 
kinesthetics and the environment. 

Eindings: Gestures/Emotions/Actions 

In an AWEDU world, avatars are rarely idle. Unless the learner specifically alters the performance settings, 
all avatars inhabiting a world will cycle through a series of gestures. The gestures are not controlled by learners, but 
instead are pre-programmed animations which consist of such movements as the folding and unfolding of arms, 
looking from side to side, looking at a wristwatch (which may not be present), shifting weight, tapping a foot, and 
patting hair. Some facial gestures include blinking and winking. Learner have no direct control over these gestures, 
but are limited in whether or not to display them on their own systems. 

Learners are only able to command a few emotions and actions of their avatar. The emotion commands might 
include such emotions as happy (the users avatar jumps up and down), and angry (pointing and shaking hand). 
Actions that can be controlled by the user include: wave, jump, and fight. These actions are not reflected from the 
learner’s perspective. 

Eindings: Avatar Point-of-View, Navigation, and Kinesthetics 

In AWEDU, the avatar serves two distinct purposes: it serves as a visual representation of the learner and it 
serves as the camera by which the learner views the 3D environment or scene. Learners have the option of selecting 
between first person perspective or an orthographic or third person perspective. The orthographic view allows users 
to see themselves (or more correctly, their visual representations) in 3D. This feature is helpful for choosing an avatar 
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and building within the world. In order for the learners to move through simulated 3D space; it is necessary for the 
avatar or camera to move. Avatar navigation is limited to moving forward and backward and side to side. Learners 
may additionally look up and down as well as fly and descend. 

There are few affordances that provide for much kinesthetic cues and experiences. Learners’ cannot control 
avatar limbs, and other than navigational movement, very little is reflected from either first-person or third -person 
perspective. While avatars do not register impact with each other, depending upon specifications designated by a 
world owner, avatars may register impact upon encountering an object. 

Analysis: Embodiment 

Learner embodiment is constructed through gestures, emotions and actions, navigation and viewing 
perspective. From the perspective of cultural kinesthetic or non-verbal behavior it is interesting to examine the 
implication of some of the gestures that animate learner avatars . In most North American cultures when engaging in a 
conversation, looking at one’s watch, or looking from side to side might indicate boredom or impatience. 

Additionally, to wink at someone may imply a range of different meanings within the context of a conversation. 
Although learners have no direct control over gestures, the non-verbal behavior conveyed through gestures may be 
in direct conflict to what is being communicated through the textbox. 

Learners have limited control over actions and emotions. They have the option of displaying an action or 
emotion; however they have no control over how the action or emotion is displayed. None of the animations are 
reflected from a first-person perspective but can only be viewed from third -person perspective. 

There are few affordances for embodiment in the AWEDU environment. For the mo st part, learners are 
prevented from displaying much non-verbal communication but must rely on text to convey emotions. This reliance 
on text might lead one to assume that the user is limited to text in constructing a sense of embodiment. Embodiment 
to some degree in AWEDU is relegated to the realm of navigation and proximity. The ability to see other learners and 
to move around may be sufficient to construct at least a limited sense of embodiment for learners . However, 
depending upon the purpose for which AWEDU is being used, it may be important to question how the limits of 
embodiment imposed on the user might limit the types of experiential knowledge that may be gleaned from these 
environments. 



Discussion and Conclusion 

One of the considerations that should be addressed by using AWEDU as a learning environment is whose 
world view is being promoted in this environment? AWEDU is embedded with cultural information and values. 
Choices made by the developers reflect and perpetuate ontological and epistemological views of the developers and 
in turn, learners in this environment are constructed by cultural values that may not reflect their own world views. 
Research into race, ethnicity, culture, and gender in virtual communities argues that virtual communities are 
embedded with values and hidden assumptions that promote the interests of some users, while marginalizing other 
users (Bailey, 1996; Balsamo, 1994; Branwyn, 1994; Cherny, 1995; Milthorp, 1996; Stone, 1995; Todd 1996; Turkle 
1995) and AWEDU is no exception. To enter this virtual environment requires one to move into Cartesian space; to 
temporarily accept the notion of a mind/body split (Pryor and Scott, 1993). This temporary acquiescence is a product 
of Western cultural values and is not shared by all cultures (Todd, 1996). The paradox of the move into Cartesian 
space is that body hidden or obscured is both liberated and repressed (Stone, 1995). Learners are both emancipated 
from physical-bound representation, yet a new construction of self is filtered through technology. 

AWEDU is a product of a Western ontology which is reflected in both how the learner and the environment 
are constructed. The affordances and constraints of the application reveal Western values both in how the learner 
and nature/environment are represented and by the lack of impact either have upon the other. The purpose of this 
investigation is not to discount the potential AWEDU might offer as a learning environment, but rather to apply a 
critical-cultural lens in which to view how this application might impact potential learners. In an era in which 
inclusion and diversity imbricate education, it is important we challenge educational developers to rethink the impact 
their designs have upon the relationship between learners and technology. Despite the constraints AWEDU imposes 
on learners one way in way in which this technology may prove most educational is that it a revealing mirror that 
reflects both values and assumptions about our culture and all it entails. 



345 




References 



Adler, P.A. & Adler, P. (1994). Observation techniques In Norman K. Denizen and Yvonna Lincoln eds. 
Handbook of qualitative research. Thousand Oaks: Sage. 

Bmckman, A. (1997). MOOSE Crossing: Construction, community, and learning in a networked virtual world 
for kids. Doctoral dissertation, Massachusetts Institute of Technology. 

Bailey, C. (1996). Virtual skin: Articulating race in cyberspace. In Mary Anne Moser with Douglas MacLeod 
(Eds.), Immersed in technology: Art and virtual environments. Cambridge MS: The MIT Press. 

Balsamo, A. (1994). Eeminism for the incurable informed. In Mark Dery (Eds.), Elame wars: Discourse of 
cyberculture. Duke: Duke University Press. 

Branwyn, G. (1994). Compu-sex: Erotica for cybernauts. In Mark Dery (Eds.) Elame wars: Discourse of 
cyberculture. Duke: Duke University Press. 

Cherny, L. (1995). “Objectifying” the body in the discourse of an object-oriented MUD. In C. Stivale (Eds.), 
Cyberspaces: Pedagogy and performance on the electronic frontier. 

De Vaney, A. (1993). Reading educational computer programs. In Robert Muffoletto and Nancy Nelson 
Knupfer (Eds.) Computers in education: Social, political & historical perspectives. New Jersey: Hampton Press. 

Dibbell, J. (1994). A rape of cyberspace; or how an evil clown, a Haitian trickster spirit, two wizards, and a 
cast of dozens turned a database into a society. In Mark Dery (Eds.), Elame wars: Discourse of cyberculture. Duke: 
Duke University Press. 

Eontana, A. & Erey, J. H. (1994). Interviewing. In Norman K. Denzin and Yvonna Lincoln eds. Handbook of 
Qualitative Research. Thousand Oaks: Sage. 

Gibson, J. J. (1977). The theory of affordances. In R.Shaw & J. Bransford (Eds.) Perceiving, acting, and 
knowing: Toward an ecological psychology (pp. 67-82). Hillsdale, New Jersey: Erlbaum Associates. 

Heim, M. (1998). Creating the virtual middle ground. Technos: a quarterly for education and technology. 7:3. 

Jeffery, P. & Mark, G. (1998). Constructing social spaces in virtual environments: A study of navigation and 
interaction. In Hook, K., Munro, A., & Benyon, D. (Eds), Workshop on Personalized and Social Navigation in 
Information Space. Stockholm: Swedish Institute of Computer Science (SICS), 24-38. 

Merchant, C. (1980). The Death of Nature: Women, Ecology and the Scientific Revolution. Mew York: 

Harper Collins. 

Milthorp, R. (1996). Eascination, masculinity, and cyberspace. In Mary Anne Moser with Douglas MacLeod 
(Eds.), Immersed in technology: Art and virtual environments. Cambridge MS: The MIT Press. 

Morningstar, C. & Parmer, P. R. (1994). The lessons ofLucasfilm’s Habitat. In Michael Benedikt (Eds.), 
Cyberspace: Pirst steps. Cambridge: The MIT Press. 

Pryor, S. & Scott, J. (1993). Virtual reality: Beyond Cartesian space. In Philip Hayward and Tara Wollen 
(Eds.), Puture visions: New technologies of the screen. London: BPI Publishing. 

Riner, R. D. (1996). Virtual ethics— Virtual reality. Putures Research Quarterly, 12 (1). 57-70. 

Stone, A. R. (1995). The war of desire and technology at the close of the mechanical age. Cambridge: The 
MIT Press. 

Todd, L. (1996). Aboriginal narratives in cyberspace. In Mary Anne Moser with Douglas MacLeod (Eds.), 
Immersed in technology: Art and virtual environments . Cambridge MS: The MIT Press. 

Turkle, S. (1995). Life on the screen: Identity in the age of the internet. New York: Simon & Schuster. 



346 




Stepping Stones to Integration: University of Northern Colorado’s Preparing 
Teachers to Use Technology Team’s Design and Development of Preservice 

Teachers 

Dennis Charsky 
Donna Ferguson Pabst 
Manisha Javeii 



Pei- Lin tin 



Caroline Sanzone 
Edward Caffarella 
University of Northern Colorado 
Abstract 

The University of Northern Colorado's (UNC) Educational Technology Department' s (ET) Preparing Tomorrow’s 
Teachers to Use Technology (PT3 ) grant proposes to infuse technology-use in the preparation of Colorado’s 
preservice teacher programs. UNC's PT3 project has five initiatives, one of which is to redesign the two, one-credit 
Department of Educational Technology courses that UNC preservice teachers are required to take. Previously, 
both courses equipped students to use computer productivity tools. Although students still learn such skills in the 
redesigned Technology in Education (200-level course), in the Educational Technology Applications (300-level 
course) they must now apply those skills to typical teaching tasks appropriate to the grade levels they are 
planning to teach. This paper discusses the processes the team followed and the stepping stones required for 
preservice teachers to be prepared to integrate technology into their K-12 classrooms. 

Introduction 

The University of Northern Colorado's (UNC) federally funded Preparing Tomorrow's Teachers to Use 
Technology (PT3) grant (Infusing Technology Use in the Preparation of Colorado Preservice Teachers) proposes to 
infuse technology-use in the preparation of Colorado preservice teachers. UNC’s grant has three overarching goals: 
(a) graduates of the UNC teacher education programs will effectively utilize technology for instruction in their 
classrooms when employed as full time teachers, (b) UNC student teachers will effectively utilize technology for 
instruction in the partner school classrooms, and (c) UNC faculty members will effectively utilize technology for 
instruction and model appropriate technology use for the preservice teacher education students. In order to meet 
these goals, the PT3 project has five initiatives, one of which is to redesign the two, one-credit undergraduate 
educational technology courses. (Initiative One: Enhance the required educational Technology courses for students 
in the Professional Teacher Education Programs [PTEP] courses.) 

Technology in Education 

The number of computers in K-12 schools has increased dramatically over the past two decades, yet 
despite this growth, computer-based technologies are not being efficiently and effectively utilized by the majority of 
teachers. Without reform to education programs we will not see improvements in technology integration in K-12 
schools (Office of Technology Assessment, 1995; Moursund & Bielefeldt, 1999). Teachers report there is a low level 
of computer use in their classrooms because of limited access to equipment and lack of training (Bosch & Cardinale, 
1993). The equipment is more than unavailable (National Center for Educational Statistics, 2001), teachers feel 
inadequately prepared to use it (AACTE, 1999; Bosch & Cardinale, 1993; Topp et al., 1995). Preservice teacher 
education schools, colleges, and departments of education (SCDEs) are not solely responsible, but they are definitely 
lagging behind in meeting the needs of new teachers who must fully understand technology integration (Walters, 
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1992). A study, commissioned by the Milken Exchange and conducted by the International Society for Technology in 
Education (ISTE), found that teacher preparation programs, while weU-intentioned, are not providing the kind of 
training and exposure teachers need if they are to be proficient and comfortable integrating technology with their 
teaching. Eurthermore, preservice teacher education candidates must be able to recognize and address the 
technology integration needs of the schools which they are ultimately placed. They must also be able to model the 
various technology usages for the teachers and students with whom they will be working. Preservice teacher 
education programs need to provide instruction in educational technology in order to better prepare prospective 
teachers (Moss, 1988). Not only does the need for technology integration surface in research, it has been mandated 
through the National Council for Accreditation of Teacher Education (NCATE) and Colorado’s Department of 
Education (CDE) content standards. 

Content and sequence of courses . The preservice teachers at UNC are now required to take both 
Technology in Education (200-level course) and Educational Technology Applications (300-level course). Originally 
these two courses were encompassed in one, two-credit class. Data gathered from students taking this course early 
in their academic program showed that the productivity tools that were taught were very helpful but the integration 
and theory components did not make a great deal of sense. On the other hand, the data showed that students who 
took this class later in their academic career felt that if they had learned the tools earlier in their coursework they 
would have used these skills to support their learning in other classes. This group of students, however, found the 
theory and integration components tremendously relevant. This two -credit course was eventually divided into two 
one-credit classes, a 200-level and a 300-level, at that point of their academic career, to better teach both the 
productivity tools and the theoretical components. Both of these courses equipped students to use computer 
productivity tools (e.g., word processing, spreadsheets), with the 300-level course adding educationally relevant 
projects. Although the classes were listed as specifically appropriate for elementary, middle, or secondary preservice 
teachers, much of the content was generic. Students typically take the 300-level educational technology courses just 
prior to student teaching at the end of their junior or beginning of their senior year. During the fall and spring 
semesters there are approximately 8 sections of the courses with 20 students in each section. Including summer 
sections, approximately 400 undergraduate preservice teacher education students enroll in the 300-level courses 
during a year. In order to meet ISTE, NCATE, and Colorado Preservice Teacher standards, as well as the objectives of 
the grant, the 300-level courses are once again being transformed. 

Lessons Learned . The grant team, consisting of the Project Director, Project Manager, and seven doctoral 
students in the Department of Educational Technology, has followed an instructional design process in the redesign 
efforts for the 300-level undergraduate courses. As part of the needs analysis, the PT3 team fully investigated 
research, state and federal standards, university curriculum, and the needs of the partner schools. Qualitative data 
gathered and analyzed by Lohr, Javeri, Mahoney, Strongin, and Gall (2000) found students taking undergraduate 
educational technology courses at UNC in the past wanted more challenging projects, project examples and non- 
examples, and fewer open laboratory meeting times. The graduate teaching assistants who served as instructors for 
these classes, felt that students who chose to take the course online and did not attend many, if any, of the classes 
had turned in less sophisticated projects than those who participated in the weekly class sessions. Lohr et al. (2000) 
also found that once students had completed the required undergraduate courses in educational technology, they 
had strong technology skills, but did not have good skills for the integration of those technologies into their 
instruction. This finding is also reflected in a national study completed by Persichitte, Tharp, and Caffarella (1998). 

ET 300-level course redesign. After implementing the redesigned 200-level courses, the PT3 team examined 
NCATE standards, CDE standards. National Education Technology Standards (NETS), International Society for 
Technology in Education (ISTE) standards, UNC requirements, and data gathered from interviews with local public 
school personnel. These data were enhanced with additional data gathered from interviews with subject matter 
experts, extant data from the partner schools, and assessment documentation. These data were thoroughly analyzed 
and objectives were written for the 300-level course to ensure UNC teacher education students would have mastery 
of the appropriate applications and integration of technology in K-12 classrooms. As a result, the PT3 team 
recognized that empowering teachers with technology tools is insufficient for preparing them to integrate technology 
into their pedagogy. Live components were identified as critical to technology integration: (a) instructional 
software/media evaluation, (b) learning theory, (c) room management, (d) data evaluation, and (e) instructional 
design. 

The graduate assistants were divided into two teams, theory and case study, based on the course content. 
The team realized, because of available data, that preservice teachers need to see technology integration modeled in 
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the teaching and learning process (Oliver, 1994; Siegel, 1994). They need to be taught how to integrate the 
technology seamlessly into the curriculum as well as understand how to use the hardware and software. 



Stepping Stones to Integration 

The theory portion of the 300-level classes addresses components that are the stepping stones to 
technology integration: Software Evaluation (SE), Rationale for Technology Integration (TI), Technology and 
Teaching Strategies (TTS), and Room Design and Management (RDM). This instruction is delivered to students in a 
variety of ways (didactic, cooperative learning, differentiated, discovery) in order to model the ways various teaching 
strategies can effectively deliver similar content through the use of technology. The culmination of the theory 
portion of the class is a set of three student created lesson plans that integrate technology. The students, however, 
are not free to produce any lesson; each student must use the Lesson Plan Options Chart (see Eigure 1). This chart’s 
purpose is to force students to create three separate lesson plans utilizing different hardware, software, content, and 
teaching strategies. The chart forces students to narrow down their options in order to devise creative methods to 
integrate technology in as many ways as possible. Each lesson plan must also address room design, state content 
appropriate standards, objectives, materials and resources, learner characteristics, and evaluation/assessment 
procedures. Because lesson plans are the backbone of a teacher’s instructions, students are carefully guided through 
an instructional design process. 

Eigure 1 Lesson Plan Options Chart 



Lesson Plan Options Chart 






Instructional Topic: 






Computers 


Available Software 


Teaching Strategies 


Choose One 


Choose One or More 


Choose One 
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Internet Explorer 






Netscape Navigator 


Cooperative Learning 




Microsoft Word 


Guided Discovery Learning 




Microsoft Excel 


Individualized Learning 




Microsoft PowerPoint 


Lecture/Demonstration 




Inspiration 


Discussion 




HyperStudio 


Other 




DreamWeaver 


(Must include typed description 


10 


Adobe Photoshop 
Content Software 
CD-ROM 
Title: 


with lesson plan.) 


30 - Computer Lab 


Internet Based 
URL: WWW. 




Mandatory Tasks 






Part A 


PartB 




One lesson plan 


Two lesson plans 




One options chart 


Two room maps 
Two options charts 





The importance of this project lies in the sequence of the four components (SE, TI, TTS, & RDM). Students 
enter the class with a plethora of emotions ranging from anxiety of working with computers to boredom and 
frustration because they do not comprehend how integrating technology can enhance pedagogy or improve student 
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learning. The basis of the Lesson Plan project is to get students excited and enthusiastic about the capabilities and 
opportunities available from integrating technology as well as giving them the foundation for the second half of the 
class, a case-based study (Sanzone, et al., 2002). 

Software Evaluation 

In order to make informed decisions about instructional software, teachers need to be aware of and 
understand the qualities of good software. Textbooks and library materials are selected carefully and judiciously in 
schools districts (Klawe, 1994); however the same is not always true with software. Most schools do not have large 
software acquisition budgets and the lack of funding becomes even more exaggerated when teachers purchase 
software without first evaluating it on the basis of specific criteria (Sanders, 1995). The first assignment for the 300- 
level courses, therefore, is based on the identification, evaluation, and selection of appropriate quality instructional 
software. An online form asks students to detail general information, system requirements, and type of application, as 
well as evaluate the software’s screen design, instructional content, orientation, motivation, navigation, and support 
available. Students complete this assignment online and also have access to an online searchable database of 
evaluated educational software for future use in their classrooms. 

Rationale for Integration of Technology 

The Rationale for Integration of Technology component answers the question we hear too often from our 
students: “Why do we have to take this computer class?’’ It is our opportunity to communicate the benefits, 
possibilities, and importance of technology integration. The objective of this component is to gain student’s 
attention by showing innovative and unique instruction using videos of best practices created by the University of 
Northern Iowa’s PT3 grant. Instruction for the class begins with a definition of technology. We define technology as 
any device that contributes or assists in instruction and/or learning. Our rationale for integrating technology is then 
presented to the class. The lecture begins with motivation theory and technology’s capability to increase students’ 
motivation through increased learner control and interactive learning environments. Next, the lecture continues by 
highlighting technology’s capability to help students visualize concepts, facts, and problems; gain instant access to 
information; and apply learning using productivity tools. Further, instructors introduce instructional environments 
and activities that aid in individualizing learning and facilitate new instructional methods (e.g. constructivism, 
collaborative learning). Finally, the preservice teachers are shown the power of technology to harness and display a 
wealth of data that can be applied to instruction and improve student learning. Next, the students see videos of 
teacher’s integrating technology into their class and pedagogy. Students are instructed to watch the video and then 
note specific instances in each video that showcase technology motivating students, using unique instructional 
methods, assists in students visualizing concepts, facts, and problems; gaining instant access to information; and 
applying learning using productivity tools, and harnessing and displaying data. After each video students are asked 
to volunteer their observations. Having sparked interest and enthusiasm, students are prepared to apply their 
excitement and knowledge. In the process, we have answered the nagging question, “Why do we have to take this 
computer class?’’ 

Technology and Teaching Strategies 

The third component. Technology and Teaching Strategies, bridges the rationale for integrating technology 
into practice. The purpose of the TTS component is to have students recognize how and where to integrate 
technology in order to enhance their traditional classroom teaching methods. This component emphasizes that 
technology use can be transparent when integrated in a manner that provides students with challenging, relevant 
tasks and activities. The class begins by discussing six teaching strategies (e.g. lecture, discussion, cooperative 
learning, individualized learning, discovery learning, hands-on learning) and how these strategies can be applied in 
traditional classrooms. 

The class then breaks into four to five small groups with each group receiving one lesson plan that 
integrates technology. The content of each technology integrated lesson plan is the United States Constitutions’ Bill 
of Rights. However, each lesson plan uses a different teaching strategy to deliver the same content 
(http://www.coe.unco.edu/ET34z/TeachingStrategy/TS_Assignment.htm). Each group then evaluates the technology 
integrated lesson on the degree to which the lesson plan fulfills the rationale for integrating technology. Further, the 
students are asked to analyze the lesson plan to judge whether the instructional objectives were met, sufficient time 
was allocated to the use of technology, and if the lesson will assist students in passing the Colorado Student 
Achievement Program (CSAP) exams. 
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During the next class, the groups are then re-created using a jigsaw technique. Students then present to 
members of the other groups, their group’s evaluation of the lesson plan they discussed. Analyzing six different 
lesson plans that use different teaching methods and various technology enables students to comprehend the 
plethora of ways technology can be integrated appropriately and seamlessly. 

Room Design and Management 

The final component. Room Design and Management, moves the students from specific theory based 
pedagogical practices to practical concerns and issues regarding technology integration. Technology can bring a 
great many benefits into the classroom but it can also cause a great deal of strife. This component (RDM) deals with 
how teachers can effectively plan for and minimize potential management problems. The preservice teachers learn 
that using job aides, schedules, student experts, and the arrangement of computers can minimize or eliminate time 
necessary for troubleshooting, correcting off task behaviors, and dealing with students accessing inappropriate 
websites. To reinforce the importance of planning ahead to minimize disasters, every student is required to evaluate 
the room arrangement of a computer laboratory (room consisting of at least twenty computers). For homework, 
students evaluate the computer laboratory, judging whether the arrangement of computers lessens sunlight glare, 
facilitates easy movement of students and teacher, and allows for easy monitoring of student efforts; evaluating 
whether or not users have the ability to adjust volume controls; and evaluating the quality of available job aids, if 
they exist. Students must also create a room map in Microsoft Word showing the computer laboratory they evaluated 
(see Figure 2). Students return to class the next week with a completed computer evaluation chart, pair up with a 
classmate, and share their room map and assessment of the computer laboratory. When students have completed the 
portion of the class, they have the necessary skills and knowledge to create three lesson plans that integrate 
technology. In subsequent classes, students enter into a problem based learning environment that utilizes each of 
these stepping stones to integrating technology. 
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Figure 2 Room Design Objects 
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Summary 

The components discussed in this paper are what the PT3 team considers the stepping stones to successful 
integration of technology. The sequence and activities in the first portion of the redesigned 300-level course 
thoroughly addresses the students’ misgivings about the required educational technology courses. By completing 
the four components, the students shed their unenlightened perspectives for a more enthusiastic creative view. The 
ET 300-level students eave this part of the class with three completed lesson plans that effectively integrate 
technology, which can be placed in their preservice teacher exit portfolio. The instruction described also prepares 
students for the second part of the 300-level class, a problem-based study that asks students to actively apply 
learned knowledge and skills (Stepping Stones) to a problem situation. Each piece of this case-based learning unit 
enhances students’ understanding of technology integration as well as the standards it is based upon. The effort to 
redesign the required 300-level educational technology course has been a year long process of analysis, design, 
development, implantation, and evaluation. Because instructional design is not a linear process, members of the 
redesign team continue to improve the class through formative and summative evaluations procedures. 
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Abstract 

Computer-based learning environments often confront learners with a number of non-embedded support devices, 
called tools. Environments in which tools are available for students assume learners to be the best judges of their 
own learning needs. However, research indicates that students do not always make adequate choices towards 
their learning process. To gain more insight in whether and how students use tools in computer-based learning 
environments, a literature review study was performed. A search was performed on the ERIC and Psycinfo- 
databases to find relevant literature with respect to tool use in computer-based learning environments. The 
publications had to deal with a research study in order to be taken into account in this review. This resulted in 25 
studies. The results of these studies are discussed in this contribution to identify underlying factors influencing tool 
use. 



Introduction 

Computer-based learning environments often confront learners with different support devices to foster 
learning. These support devices can either be embedded or non-embedded support devices. Embedded support 
devices are totally integrated in the learning environment and cannot but be used by learners. Examples of such 
devices are feedback or structure in learning material. Non-embedded support devices or tools on the other hand are 
support devices which use depends on the initiative of the learners. They are added to the environment, and the 
learner decides on their use. In this contribution the latter kind of support devices is addressed. 

However, it should be noticed that not all authors identify these support devices as tools, as devices whose 
use depends on the initiative of the learner. Eor instance, Hannafin, Land and Oliver (1999) make a distinction 
between tools and scaffolds, where scaffolds relate to the explicit guidance of the students and tools provide the 
means through which individuals engage and manipulate both resources and their own ideas. In this contribution 
scaffolds are identified as tools whenever their use has to be initiated by the learners. Moreover, all other categories 
could also be embedded support devices if the initiative to their use would not rest with the learner. 

Designing environments in which tools are available for students assumes learners to be good judges for 
their own learning needs. They select the tools whenever they need it. Students are in control over the use of these 
tools. Contrary to these assumptions, research on learner control, reveals no clear benefit of learner control on 
learning (see reviews of Large, 1996; Eriend & Cole, 1990; Goforth, 1994, Williams, 1996). In most learner control 
studies a positive effect on learner's attitude was found. The effect on learning results is clearly mediated by various 
student characteristics. Basically these reviews conclude that students are usually not capable to make adequate 
choices for themselves (see also Land, 2000; Hill & Hannafin, 2001; Clark, 1991; Lee & Lehman, 1993; Milheim & 
Martin, 1991), i.e. choices beneficial for their learning process. In an instructional context, students seem to lack the 
necessary skills to monitor and regulate their own learning. It is likely that similar problems occur with respect to 
control over tool use. If students indeed lack monitoring and regulation skills, they will have problems in determining 
when they need help, what kind of help they need and hence, when the use of specific tools might be beneficial. A 
recent evaluation study and follow up study performed by the authors for instance, confirmed that students do not 
use the available tools when working on a diagnostic problem in a computer-based training program (Clarebout, Elen, 
Lowyck, Van den Ende and Lagana, 2000; Clarebout, Elen, Lowyck, Van den Ende, Van den Enden, van Compel & 
Vermeulen, 2001). Additionally, if students used the tools, they did not use them in an optimal way. The problem 
seems not to be specifically related to tools. Eor instance, Greene and Land (2000) reveal that even when support 
devices are integrated in the environment and students are obliged to use them, they tend to use them inadequately. 
Questions were provided to students to engage in deeper processing. In contrast to using these questions as a tool 
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to aid cognition, students ignored them or responded with activities rather than the different underlying processes. 
Bmsh and Saye (2001) came to the same results in their study. 

From the above, it can be noticed that the use of tools cannot be taken for granted. In order to gain more 
insight in the use of tools, a literature review study was performed. This study deals with tool use in computer-based 
learning environments. Consequently, only studies with computer-based learning environments are discussed. Do 
students use tools and if so, how do they use them is the core question. The results of this review may provide more 
empirical insights on the different assumptions underlying tool use and may help to identify learner characteristics 
and other variables playing a role with respect to tool use. 

After a brief discussion of the methodology, an overview of the results is presented. Next, the results of 
these studies are reflected on in the discussion and finally possible solutions and suggestions for further research 
are offered in the conclusion. 



Method 

This literature review started with a search on the ERIC and Psycinfo -databases (for the descriptors: see 
Table 1). In Table 1, the number of initially found records is presented. The descriptors relate specifically to the use 
of tools or synonyms (options, adjunct aids) and to environments in which tools are most likely to be available (open 
learning environments, hypermedia environments). No limitation was applied for publication year. For all searches, 
the term "research" was added, since the aim was to find research studies involving tool use rather than mere 
descriptions of tools. If the search yielded too many results (N > 300), "computer-i-" and "use” were additionally 
entered as descriptors. Since this still resulted in 694 results for the ERIC database and 407 for the Psycinfo database 
for the descriptor ''simulation-i-'', the descriptor "support" was added to reduce the amount of results. 



Table 1: Descriptors 



Descriptors 


Number of initial 
documents 




ERIC 


Psycinfo 


“option-i- use” and research 


4 


4 


“use of option-t’ and research not “option-i- use” 


10 


3 


“use of tool-t” and research 


54 


161 


“tool-i- use” and research not “use of tool-t” 


17 


0 


“open learning environment-t” and research 


5 


5 


“electronic learning environment-i-” and research 


7 


1 


“hypermedia environment-i-” and research 


10 


14 


hypermedia and research and use not “hypermedia 
environment-i-” 


298 


165 


“learner control” and research 


204 


60 


“instructional intervention-t” and research 


64 


35 


“adjunct aid-i-” and research 


4 


1 


“discovery learning” and research and computer-i- 


95 


44 


“use of resource-i-” and research 


119 


52 


“inquiry -oriented instruction” and research 


9 


0 


“project-based” environment and research 


3 


2 


“computer-assisted learning” and research 


108 


25 


Simulation-i- and research and use and support 


61 


48 



All abstracts were read to find out whether these publications dealt with research on the use of tools. If 
these studies involved the use of tools, a research study (quantitative or qualitative) and a computer-based learning 
environment, the article was withdrawn. No constraints were put on level of education. Additionally, references of 
articles were looked at to find more studies. If a reference was found to a conference paper, the author was contacted 
to request that paper, accompanied by a request for similar references. Eventually 25 studies were withdrawn. 

Results 
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To present the results, a distinction is made between the use of tools as independent, dependent and 
mediating variable. These terms should be considered in a broad sense. Some of the studies deal with correlations 
rather than causal relationships. This means that in a strict sense one cannot speak of dependent or independent 
variable. Depending on the aim of the study or the timing of events, the results were classified in one of the different 
sections. 

Use Of Tools As Independent Variable 

In this part, studies will be reported dealing with the use of tools as an independent variable. This will give 
insights in the effects of adding tools on the learning process. 

Main Effects 

Dempsey and Van Eck (1998) found that using an advisor tool positively correlated with the scores on the 
post-test. Unfortunately, the restricted number of times the advisor was consulted disabled them to apply causal 
statistics. 

Viau and Larivee (1993) revealed that the use of a glossary explained 21.6% of the variance of results on 
performance. The use of a navigation map on the other hand did not influence the performance. 

Carrier and Williams (1988) divided students in high option users and low option users and studied the 
difference between those two groups for performance. Only a moderate relationship was found between the level of 
option selection and performance. 

Oliver and Hannafin (2000) researched the effect of tool use on higher order thinking. No influences were 
found. This might be explained by referring to the tools students used. The analysis revealed that mainly procedural 
tools were used and that tools aiming at helping the problem solving process were only seldom used. Additionally, 
qualitative analysis indicated that if these tools were used, they were not used as they were supposed to be. They 
were developed to scaffold higher order thinking, but students did not engage in higher order thinking processes 
when using these tools. Eor instance, they did not use the tools to organise information or to justify ideas, instead, 
when they used them, they applied them to make lists of web pages. 

Interaction Effects 

Carrier, Davidson and Williams (1985) compared students working either with a program in which students 
could decide on the use of elaboration tools (lean version) or with a program where students could choose to bypass 
elaborations (full version). They compared high ability users in the lean versions using a high number of options to 
high ability and low ability students in the full version to determine whether students made 'good' or adequate 
choices. High ability students seemed to profit from the choice they had in the lean version. However, some problems 
arose with the statistics since only few higher ability students selected few options and only few lower ability 
students selected many options in the lean version. Viau and Larivee (1993) found that for high ability students, time 
on and frequency of consulting the glossary was a better predictor for performance than time on the subject matter. 
The frequency spent on the navigation map had no influence. In the study of Carrier and Williams (1988) extremely 
high and extremely low option selectors or tool users showed decrements in the program controlled condition. 

Cohill and Williges (1982) studied the effect of user versus computer initiated and selected help. Help 
initiated and selected by the user resulted in less time to finish the task, fewer errors and fewer editing commands, 
than help initiated and / or selected by the computer. 

Martens, Valcke and Fortier (1997) revealed an interaction effect between discernability and the use of 
processing tools on study outcome. When support is discernable, students seldom using processing tools have a 
higher score on the post-test than students frequently using those tool. Students seldom using processing tools 
score lower on the post-test than students using these tools more frequently, when support is non-discernable. 

Summary 

Some studies revealed a (moderate) positive effect of tool use on performance (Dempsey & Van Eck, 1998; 
Viau & Larivee, 1993; Carrier & Williams, 1988). This effect seems to be mediated by the kind of tool (e.g. use of 
glossary versus use of navigation map) and by the way students use these tools (Oliver & Hannafin, 2000). 

There is some evidence for an interaction between ability and tool use on performance (Carrier et al., 1985). 

It should be considered however, that most of these studies only deal with the effect of tool use on 
performance as a learning product. The influence on learning processes is only addressed in one study. Moreover, 
that was also the only study in which it was not merely looked at the quantity of tool use, but also the quality of tool 
use by looking at how students actually use the tools (Oliver & Hannafin, 2000). Table 2 provides an overview of the 
different studies and their results. 
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Table 2: Tool use as independent variable 

N Environment dependent Result 

variable 

Main effects 



Dempsey & Van 
Eck, 1998 


58 


Computer-based 

lesson 


Performance 
during instruction 


Pearson correlation: r = .407 p<.05 








Time spent on 
instruction 


Pearson correlation: r = .432 p<.05 








Television 

watched 


Pearson correlation: r = .292 p<.05 


Viau & Larivee, 1993 


70 


Interactive 

textbook 


Performance 


Multiple regression analysis: 

21.6% of variance explained by frequency of 
glossary consultation. 

Significant contribution of time (r = .32) and 
frequency (r = .44) of glossary consultation 
No significant contribution for time (r= .08) 
or frequency ( r = .12)of navigation map 
consultation 


Oliver & Hannafin, 
2000 


12 


Problem-based 

learning 

environment 


Use of higher 
order skills 


Qualitative analysis: No effects found (little 
use) 

=> no descriptives provided 


Carrier & Williams, 
1988 


114 


Computer-based 

lesson 


Performance 


Pearson correlation of option selection and 
post-test: r = .29, p<.05 ; with delayed test: r 
= .20 po05 

Low opt. sel.: Mp= 4.7 SD=2.8 / Md= 4.8 
SD=3.1 

Medium-low opt.sel.: Mp=5.6 Sd=2.9 / 
Md=5.0 SD=3.4 

Medium-high opt.sel.: Mo=7.4 SD=3.0 / 
Md=6.4 SD=3.4 

High op.sel.: Mp=6.9 SD=3.7 / Md=6.5 
SD=3.8 



Interaction effects 



Carrier et ah, 1985 


20 


Computer-based 


Performance 


Fisher’ s Exact test: 






course 




High-ability high option: post-test = .04 
delayed =.11 

High-ability low option: post-test:. 80 
delayed: .80 

Low ability high option: post-test:. 38 
delayed:.51 

Low ability low option: post-test: .11 
delayed:. 34 


Cohill & Williges, 


72 


Performance 


Time finishing 


User initiated user selected: M = 332.0 


1982 




task 


task 


Other help configuration: M=394.8 








Errors per task 


User initiated user selected: M = 0.8 
Other help configuration: M=L7 








Numbers of 


User initiated user selected: M = 11.0 








commands 


Other help configuration: M=15.0 








Command ratio 


User initiated user selected: M = 2.7 
Other help configuration: M=6.3 
ANOVA’s: no statistics presented, SD not 
given 




















































Viau & Larivee, 1993 


70 


Interactive 

textbook 


Performance 


Regression analysis: Frequency glossary 
consultation: 

Weak: M = 8.33 SD = 6.77; r = .43 p<.05 
Average: M = 11.36 SD = 9.08; r = .50 p<.05 
Strong: M= 10.75 SD=6.63; r=.30, p<.05 
Time on glossary: 

Weak: M=1 1.46 SD =1 1.29; r= .26 p<.05 
Average: M=16.26 SD = 16.07; r=.39 p<.05 
Strong: M=12.11 SD =7.62; r=.54 p<.05 


Carrier & Williams, 
1988 


114 


Computer-based 

lesson 


Performance 


MANOVA on repeated measures: Interaction 
between treatment and quadratic level of 
choice: F(l,86) = 4.28, p<.05 
Interaction time by level of option selection, 
quadratic trend interaction: F(l,86)=4.83 
p<.05 


Martens et al. 1997 


43 


Interactive 

textbook 


Performance 


Discemability and processing tool: 
ANOVA: F(l,42) = 5.66, p<.05 
Discemability and testing tool: 
ANOVA: F(l,42)=3.67, n.s. 
Discemability and orientation tool 
ANOVA: n.s.; no results provided 



Use Of Tools As Dependent Variable 

Studies on the use of tools as a dependent variable can be divided by considering the independent 
variables: student characteristics, support attributes, working method and additional advice. Interaction effects are 
dealt with after discussion of the main effects. These studies may contribute to identify variables that influence tool 
use. 

Some studies refer to the same studies as presented in the previous section when use of support was both 
dependent and independent variable. 

Student Characteristics: Main Effects 

The following student characteristics have been investigated in the retrieved studies: general ability, prior 
knowledge, learning style, epistemic beliefs and metacognitive awareness. 

The study of Carrier et al. (1985) showed high ability students using more elaboration tools than low ability 
students. Schnackenberg and Sullivan (2000) could not confirm these results, as they did not find any differences 
between low and high ability students. Chapelle and Mizuno (1989) did not find any significant difference for amount 
of tool use per minute, but did find a difference for amount of tool use per sentence. Low ability students use more 
tools per sentence than high ability students do. A more detailed look indicates that this difference is accounted for 
by consultation of facts and not by consultation of a grammar tool or dictionary. Chapelle and Mizuno also revealed 
high ability students to use tools differently than low ability students. High ability students use tools more while 
solving problems whereas low ability students use tools more as advance organizers. 

The influence of prior knowledge on the use of tools was studied as well. Martens et al. (1997) found a 
positive effect of prior knowledge on tool use: the more prior knowledge students posses the more tools they use. 
Viau and Larivee (1993) claim this relation to be curvilinear. In their study, average students used additional 
information more frequent than both weak and strong students did. 

Learning style was defined as the difference between active, neutral or passive learners (Lee & Lehman, 
1993, Dempsey & Van Eck, 1998) or as field (in)dependence (Liu & Reed, 1994; Carrier, Davidson, Higson & Williams, 
1984). No main effects of learning style defined as passive, neutral or active learners could be found. Neither did Liu 
and Reed (1994) find an effect for field (in)dependence in their study on the influence of learning style on the 
consultation of different tools. Although not significant, there seems to be a tendency for field independent students 
to more frequently use the index tool than field dependent students, while the mixed group more frequently uses the 
note -taking tool. 
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In contrast to Liu and Reed (1994), Carrier et al. (1984) did find an effect of field (in)dependence, but only for 
one elaboration tool, namely viewing additional examples. Field independent students viewed more additional 
examples than field dependent students did. For the other three tools (definitions, practice instances and feedback) 
no differences were found. Hasselerharm and Leemkuil (1990) found no difference for field (in) dependence. 

Hartley and Bendixen (2001) studied the effect of metacognitive awareness and epistemic beliefs on the use 
of comprehension aids such as objectives and glossaries in a hypermedia tutorial. No effects were found of 
metacognitive awareness. Some significant correlations were found between belief in innate ability as epistemic 
beliefs and the use of self-check questions, belief in quick learning and respectively the use of advanced organizers 
and glossary. Opposite to what was expected, learners with more naive epistemic beliefs seemed to use more tools 
than learners with more sophisticated beliefs. 



Tool Attributes: Main Effects 

Martens et al. (1997) studied the effect of discernability of support on the use of tools. Besides the 
interaction effect as earlier reported, the results of their study indicate no main effect of discernability on the use of 
tools. Dempsey and Van Eck (1998) found an advisor tool presented on screen resulting in a higher consultation than 
an advisor tool as part of a pull-down menu. 

A large group of studies in this category deals with the comparison of fullminus and leanplus conditions 
(Hannafm & Sullivan, 1995; Crooks, Klein, Jones & Dwyer, 1996; Crooks, Klein &Savenye, 1998; Hicken, Sullivan & 
Klein, 1992; Schnackenberg & Sullivan, 2000). In these studies, a comparison is made between students who can 
bypass instruction (fullminus) or students who can actively select more instruction (leanplus). The additional 
instruction consists of reviews, summaries and practice items (elaboration tools). In all these studies, significantly 
more instruction is selected in the fullminus group than in the leanplus group. 

Crooks et al. (1998) found however that this difference in use of elaboration tools can be attributed to one 
function. In their study, they found overall difference between the leanplus and fullminus group to be related to the 
consultation of practice items. No differences were found between the two groups for the other elaboration tools. 

This seems to be an indication that the kind of tool matters. Similarly, Carrier, Davidson, Williams and Kalweit (1986) 
found a difference in use between different tools: paraphrased definition, expository instances, practice instances 
and analytic feedback. Paraphrased definitions were chosen more often than expository instances. Practice items 
were also more frequently chosen than expository instances. 

Next to these studies, a number of studies report similar findings although no real main effects were found. 
These studies did not explicitly measure whether the difference in use was significant. Oliver and Hannafm (2000) 
provided students with different kinds of tools: procedural, cognitive tools, and scaffolding tools. Students used 
almost exclusively the procedural tools. Similarly, Fischer, Troendle and Mandl (in press) found that all students used 
a visualization tool, but only a limited number of students used the knowledge resources. Nevertheless, a prior 
knowledge test indicated that all students might have gained benefit from using these resources. The human tutor on 
the other hand was consulted very frequently for technical functioning and organization of collaboration. 

Working Method: Main Effects 

Crooks et al. (1996; 1998) investigated the influence of individual versus co-operative work on the use of 
help tools. The 1996-study, revealed individuals to more frequently use optional elements than co-operative groups. 
In the 1998-study however, no differences were found between the two working methods. 

Additional Advice Or Training: Main Effects 

Lee and Lehman (1993) studied whether presenting instructional cues to students working in a hypermedia 
environment would result in a more frequent consultation of information. A main effect was found. Students who 
receive additional cues select more information than without any cues. Grasel, Fisher and Mandl (2000) found 
students receiving strategy training to make more adequate use of additional information available through a 
glossary, a diagnostic help tool and a database. They adapt their problem solving process based on this information. 
The group without strategy training ignored this information. 

Relan (1995) investigated the effect of training students' tool use on its consequent use. Students had 
access to three kinds of reviews, immediate review (after each section), global review (after a whole session) and 
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practice review. Students with training reviewed more before practice than during practice, which was not the case for 
students without training. An ANOVA with amount of review before practice as dependent variable and training as 
independent variable did not show significant effects. Carrier et al. (1986) compared a group that received 
encouragement to use the options to a group that did not receive this encouragement. The encouragement treatment 
group selected more options than the no-encouragement group. 

Interaction Effects 

Hannafin and Sullivan (1995) found an interaction effect between ability and version of the program. In a 
leanplus version of the program, high ability students selected more (43%) options than low ability students (19%) 
did. This difference was not found for the fullminus version of the program. Lee & Lehman (1993) found an 
interaction of the treatment condition with learning style. Learners with a neutral (versus active or passive) learning 
style with instructional cues selected more information than learners with the same learning style without 
instructional cues. 

Relan (1995) reveals an interaction between treatment group and training. Students in a limited learner 
control condition review more items when they receive training than students in the complete learner control 
condition when they receive training, although the latter group had more options to review. 

Summary 

Some evidence is found that ability and prior knowledge affect the amount of tool use. Ability also seems to 
influence the quality tool use. No evidence was found of learning style influencing the use of tools, except for an 
interaction effect with the provision of instructional cues. It could be said that there is a relationship between the use 
of some tools and epistemic beliefs, but a closer look reveals low (though significant) correlations (see Table 3). None 
of the correlations exceeds 0.3. These studies highlight the kind of tool influences the use. For instance, performance 
support tools or procedural tools are more frequently used than cognitive or scaffolding tools. 

The results on the effect of working method do not give the opportunity to draw any conclusion. 

Advisement, instructional cues and encouragement on the other hand affect both quantity and quality of 
tool use. Training does not result in clear effects. An overview of the results is presented in Table 3. 
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Table 3: Tool use as dependent variable 





N 


Environment 


Independent 

variable 


Result 


student characteristics 


Carrier, et al. 
1985 


28 


Computer- 
based lesson 


Ability 


Path analysis: partial B coefficient: .59 
No descriptives 








Locus of control 


Path analysis: partial B coefficient: -.08 
No descriptives 


Schnackenberg & 
Sullivan, 2000 


99 


Computer- 
based lesson 


Ability 


Amount of optional screens^ 
High: M = 25.04 SD = 15.57 V 
Low: M = 20.15 SD = 15.68-^ 


ANOVA: F(l,98) = 3.71, n.s. 


Viau & Larivee 
1993 


70 


Interactive 

textbook 


Prior knowledge 


Amount of tool use: Regression 
Weak : M = 8.3 SD = 6.8 r =.43 
Average :M=11.4SD = 9.1r = 
Strong : M = 10.8 SD = 6.6 r =.i 


analysis 
p<.05 
.50 p<.05 
0 p<.05 


Chapelle & 
Mizuno 1989 


13 


Computer- 
based lesson 


Prior knowledge 


Amount of tool use per 
minute 

High: M = .09/min. SD = .06 
Low: M = .14/min SD = .08 
O T-test: t = -1.34; df=ll, 
n.s. 

Amount of tool use per 
sentence 

High: M = .13/sent SD = .09 
Low: M = .28/sent SD = .14 
■=> T-test: t = -2.27 df= 11, 
p<.05 


Purpose of tool use 
On-going problem solving: 
High = 78% Low = 48% 
Advanced organizer 
High = 7% Low = 32% 
Reconfirmation 
High=ll% Low = 9% 
Extra information 
High = 0% Low = 7% 
Others 

High = 4% Low = 4% 


Martens et al. 
1997 


51 


Interactive 

text-book 


Prior knowledge 


MANOVA: ? = 0.88, p< .05 
Prior knowledge enhancing effect on use 
processing tools F(l,50) = 6.12, p<.05 
testing tools E(l,50) = 3.99, p<.05 
orienting tools: not significant , no E-value 








Reading 

comprehension 

skills 


MANOVA: 

No significant effects / descriptives, no statistics 


Lee & Lehman, 
1993 


162 


Computer- 
based lesson 


Learning style 

(active/neutral/passi 

ve) 


Selection frequency: 

Active: M = 0.86 SD = 0.661 
Neutral: M = 0.82 SD = 0.72J 
n.s. 

Passive: M = 0.58 SD = 0.62 


ANOVA: F(2,161) = 2.64, 


Dempsey & Van 
Eck, 1998 


58 


Computer- 
based lesson 


Learning style 

(active/neutral/passi 

ve 


Amount of tool use 
Passive: M = .86 SD = 2.07 1 
Neutral: M = 3.23 SD = 3.23 J 
Active: M = 2.56 SD = 5.06 


ANOVA: n.s. 

(statistic not presented) 


Liu & Reed 1994 


63 


Computer- 
based lesson 


Learning style (field 
(in)dependence) 


Amount of total tool use 
FD: M = 16.21 
Fmixed: M = 29.28 
FI: M: 24.84 

ANOVA’s for 5 support tools: n.s. , results reported for use of 
index: F(2,62) = 2.54, p = .09, no results reported for other 4 
tools 


Carrier et al. 
1984 


44 


Computer- 
based lesson 


Learning style (field 
(in)dependence) 


Frequency of option selection (proportion of total) 

Definitions: FI: .25; FD: .22 
Expository instances: FI: 017; FD: .25 
Practice instances: FI: .25; FD: .20 
Feedback: FI: .34; FD: .35 

=> Chi-square analysis: only difference for expository instances: 
ChP = 8.87, p<.05 
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Hasselerharm & 
Leemkuil 1990 


110 


Computer- 
based lesson 


Learning style (field 
(in)dependence) 


45% used advisement; no significant effect. No statistics 
provided 








Prior achievement 


No significant effect, no statistics provided 


Hartley & 
Bendixen, 2001 


101 


Computer- 
based lesson 


Metacognitive 

awareness 


Regulation of cognition; Correlation with: use of: 

AO: r = -.008, p>.05 / O.: r = .009, p>.05 / G: r = -.086, p>.05 / 
SM: r = -.044, p>.05 / SC: r = .06, p>.05 
Knowledge of cognition; Correlation with: use of: 

AO: r = -.144, p>.05 / O: r = -.028, p>.05 / G: r = .011, p>.05 / 
SM: r = -.1 13, p>.05 / SC: r = -.086, p>.05 








Epistemic beliefs 


Omniscient authority: Correlation with use of 

AO: r =-.091, p>.05 / 0: r = .002, p>.05 / G: r = .077, p>.05 / 

SM r = -.032, p>.05 / SC: r = -.082, p>.05 

Innate ability: Correlation with use of: 

AO: r = -.016, p>.05 / O: r = .125, p>.05 / G: r = .032, p>.05 / 
SM: r = -.007, p>.05 / SC: r = .271, p<.05 
Simple knowledge: Correlation with use of: 

AO: r = .04, p>.05 / O: r = .052, p>.05 / G: r = .141, p>.05 / SM: 
r = .066, p>.05 / SC: r = .043, p>.05 
Quick learning: Correlation with use of: 

AO: r = .183, p<.05 / O: r = .09, p>.05 / G: r = .190, p<.05 / SM: 
r = -.163, p>.05 / SC: r = .163, p>.05 
Certain knowledge: Correlation with use of: 

AO: r = .062, p>.05 / O: r = -.021, p>.05 / G: r = -.008, p>.05 / 
SM: r = -.015, p>.05 / SC: r = .054, p>.05 


Support attributes 


Hannafin & 
Sullivan, 1995 


133 


Computer- 
based course 


Leanplus/Fullminus 


Amount of total-optional screens consulted 
Leanplus: 32% L 

Fullminus: 78%J ANOVA: F(l,132) = 4.13 p<.05 


Crooks et al. 
1996 


125 


Interactive 

textbook 


Leanplus/Fullminus 


Amount of total optional screens consulted 
Leanplus: 56% L 

Fullminus: 83%J ANOVA: F(l,124) = 51.96, p<.05; ES = .99 


Crooks et al. 
1998 


97 


Interactive 

textbook 


Leanplus/Fullminus 


Amount of total optional screens consulted 
Leanplus: M = 67.36 SD = 16.M 

Fullminus: M = 82.70 SD = 9.li" ANOVA: F(l,96) = 34.36, 
p<.05 


Hicken et al. 
1992 


111 


Computer- 
based course 


Leanplus/Fullminus 


Amount of total optional screens consulted 

Leanplus: 32% (SD = 271 ANOVA: F(l,92) = 70.80, p<.05; 

Fullminus: 80% (SD = 25J ES = 1.79 

O 


Schnackenberg & 
Sullivan, 2000 


99 


Computer- 
based course 


Leanplus/Fullminus 


Amount of total optional screens consulted 
Leanplus: 35% L 

Fullminus: 68%J ANOVA: F(l,98) = 30,42, p<.05; ES = 1.08 


Martens et al. 
(1997) 


120 


Interactive 

textbook 


Discernability 


ANOVA: No significant effects (no statistics presented) 

This group partly involves students who have a printed version 
of the textbook 


Carrier et al. 
1986 


37 


Computer- 
based lesson 


Option type 


Amount of times selected: 

Paraphrased definition: M =62.8 SD = 37)l 
Expository instances: M = 26.1 SD = 25.9^ ANOVA: 

Practice instance: M = 31.5 SD = 30. I F(3, 105) = 

20.13, 

Analytic Feedback: M = 34.5 SD = 25.9 p<.05 


Working method 


Crooks et al. 
1996 


125 


Interactive 

textbook 


Individual/ co- 
operative 


Amount of total optional screens consulted 

Individual group: M = 74% ANOVA: F( 1 , 1 24) = 4.92, 

Co-operative group: M = 65% p<.05, ES = .35 


Crooks et al. 
1998 


97 


Interactive 

textbook 


Individual/ co- 
operative 


Amount of total optional screens consulted 
Individual group: M = 75.14 SD = 12.99 
Co-operative group: M = 74.92 SD = 12.31 
■=> ANOVA: Not significant (no statistics presented) 



Additional advice or training 
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Instructional cues 



Lee & Lehman, 162 Computer- 
1993 based lesson 



Grasel et al. 2000 24 Problem-based Strategy modeling 

learning 
environment 

12 Instruction 



Relan (1995) 109 Computer- training 

based lesson 



Carrier et al. 
1986 


37 


Computer- 
based lesson 


encouragement 


Interaction effects 


Hannafin & 
Sullivan, 1995 


133 


Computer- 
based course 


Ability -i- 

Leanplus/Fullminus 


Lee & Lehman 
1993 


162 


Computer- 
based lesson 


Learning style -l- 
instructional cues 


Relan 1995 


109 


Computer- 
based lesson 


Learner control + 
training 



Selection frequency 

With cues: M = .94 SD = .58 ~L 

Without cues: M = .59 SD = .?3 ANOVA: F(l,161) = 9.81, p 

<.05 

Amount of using additional information corrective 
Strategy group: M = 15.88 
Without strategy group: M = 9.13 
O Mann-Whitney U-test: U = 8.0 p<.05 
Amount of using additional information corrective 
Instruction group: M = 8.17 
Without instruction group: M = 4.83 
O Mann-Whitney U-test: U = 8.0 p<. 10 
Total amount of review: 

Learner control complete LCC: 

Comprehensive training: M: 3.2 SD = 2.4 
Partial training: M = 2.9 SD = 3.0 
No training: M = 3.1 SD = 2.8 
Learner control limited (LCL) 

Comprehensive training: M= 4.2 SD = 3.6 
Partial training: M = 1.2 SD = 1.6 
No training: M = 2.4 SD = 3.6 
O ANOVA: Not significant within LCC 
O ANOVA: Not significant over two groups (no statistics 
presented) 

Amount of option use 
Encouragement group: M = 38.8 
No encouragement group: M = 25.4 

O ANOVA: F( 1,35) = 4.82, p<.05 

Amount of additional screens 

Fullminus- Low ability: M = 76% SD = 18.7% 

Fullminus- High ability: M = 79% SD = 20.6% 

Leanplus-Low ability: M = 19% SD = 5.1% 

Leanplus_High ability: M = 43% SD = 1 1.4% 

'=> ANOVA: F(l,132) = 12.36 p<.05 

Selection frequency 
With cues-Active: M = .082 SD = 0.52 
With cues-Neutral: M = 1.17 SD=0.58 
With cues-Passive: M = 0.75 SD = 0.53 
Without cues-Active: M = 0.90 SD = 0.78 
Without cues-Neutral: M = 0.47 SD = 0.67 
Without cues-Passive: M = 0.72 SD = 0.68 

O ANOVA: F(2,161) = 5.55 p<.05 

Total amount of review during practice: 

Means see above 

■=> ANOVA: F( 1 ,94) = 6.50, p <.05 



Use Of Tools As Mediating Variable 

This part discusses studies that try to explain differences between groups by referring to the amount of tool 
use. Explorative studies that cannot draw firm conclusions due to the lack of tool use are included as well. Previous 
sections already mentioned some of these studies (e.g. Dempsey & Van Eck, 1998). 

Some of the studies in which a fullminus and leanplus group are studied found either no effect between 
these groups on performance (e.g. Hannafin & Sullivan, 1995; Crooks et al., 1998) or they found the fullminus group 
to outperform the leanplus group (e.g. Crooks et al., 1996; Schnackenberg & Sullivan, 2000). This finding is explained 
by referring to the number of tools used, which in all those studies is lower for the leanplus group than for the 
fullminus group. In a similar effort, Morrison, Ross and Baldwin (1992) compared a learner control to a program 
control group. The learner control group performed worse than the program control group. Apparently, the learner 
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control group had reviewed only a limited number of items. From the possible total of 12 review items, 42.3% reviewed 
3 items, 40% four or five items and only one person reviewed all items (N = 73). 

Carrier et al. (1985) investigated through means of path analysis whether the 'tool choice' mediates between 
student characteristics and scores on tests. There was a significant positive correlation between tool use and post- 
and delayed test score, but when controlling for ability, this correlation disappeared. 

Overall, the studies provide no clear empirical evidence about the mediating role of the variable tool use. 
Nevertheless, different effects of the learner and program control seem to be explained by referring to differences with 
respect to tool use. It should also be noted that a number of studies could not be finalized because students did not 
very often use the tools put to their disposal (see also previous sections). 

Discussion 

Merely providing tools to students does not necessarily result in (adequate) use of tools (e.g., Dempsey & 
Van Eck, 1998; Oliver & Hannafm, 2000, Fisher, Troendle & Mandl, in press). Studies investigating the effect of tool 
use do not reveal clear effects on performance. Interaction-effects and studies in which tool use is a dependent 
variable suggest that the lack of effect is due to learner characteristics such as ability and prior knowledge. In the 
introduction, the lack of monitoring and regulation skills was mentioned as an explanation of limited tool use. While 
metacognitive skills are acknowledged by different authors to be essential to make adequate learning decisions (de 
Jong & van Joolingen, 1998; Land, 2000; Hill & Hannafin, 2001, Lee & Lee, 1991, Kinzie & Berdel, 1990), only one 
study deals with the influence of metacognition on tool use. Hartley and Bendixen (2001) address metacognition, and 
more specifically metacognitive awareness. This metacognitive awareness was measured by the Metacognitive 
Awareness Inventory (Schraw & Dennison, 1994, in Hartley and Bendixen, 2001) and classifies items into two 
categories of metacognition, knowledge of cognition and regulation of cognition. Whether measuring metacognitive 
skills by means of a survey is a good representation of the skills students apply when dealing with a learning task 
might be. 

A student characteristic that was not considered in the different studies, but might be of relevance to tool 
use are students' beliefs or perceptions about these tools. Winne (1985) already indicated that the functionality 
students ascribe to certain elements in a learning environment influence whether and how they will use these 
elements. Findings of Clarebout, Elen, Lowyck, Van den Ende and Lagana (2000) for instance, revealed students to 
think they were cheating when they consulted tools (cognitive tools and scaffolds). This might exp lain the limited 
use students made of the available tools. Interviews in the study of Brush and Saye (2001) clearly show students not 
using the tools as intended since students were unsure how to use them. This may also explain why the provision of 
additional advice results in positive effects. 

In addition to learner characteristics, the kind of tools seems to influence the amount of use as well. 
Procedural tools are use more frequently than cognitive tools or scaffolds. Results of studies with elaboration tools 
indicate also that students used some tools more often than others (e.g. more practice items than reviews) or that 
effects are found for specific tools and not for others. 

Overall however, the studies do not clearly reveal what variables influence tool use or what the effect is of 
tool use. The following observations may explain this lack of clear insights: 

• In most studies the mere frequency of consultation of tools is looked at rather than the adequacy of the 
use. In order to take the latter into account, not only the amount of use should be considered, but also 
what students actually do when they use tools (e.g., Grasel et al., 2000). A problem inherent to this 
aspect is that students should first use the tools before it can be studied how they use the tools. The 
section on tool use as mediating variable indicates that this is not always the case. 

• The reported studies mostly deal with course-based environments. Students study a lesson that is 
presented on a computer and afterwards they get a knowledge test. It might be questioned whether 
students have a need to consult the tools to improve their learning process or performance with respect 
to the specific task. 

• Some studies show that the lack of using tools is related to lower performance on a learning task 
(Dempsey & Van Eck, 1998), or that consulting a help function explains a significant proportion of the 
variance on performance (Viau & Larivee, 1993). Only a limited number of studies directly address the 
relation between the use of tools and performance (see limited number of studies in section 2.1). Oliver 
and Hannafin (2000) aimed at studying this effect, but could not draw any conclusion since students 
hardly used the tools. If the effects of tool use on performance are studied, only learning results are 
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looked at. The learning process itself and the effects of using these tools on the learning process are 
addressed in only one study. 

• Support tools mostly provide additional support on the content level (elaboration tools). In a few 
studies only support on other levels is provided such as on a problem solving or metacognitive level 
(e.g., cognitive tools, scaffolds). This can also be due to the course-based character of the 
environments. 

• A more methodological remark relates to the research instmments. Overall, studies use only one 
instrument to collect data, while a triangulation of different instruments can provide a closer insight in 
the amount, the timing and the quality of tool use. For instance, a combination of log files, surveys and 
observations could provide a more complete picture of students' learning process when working in a 
computer-based learning environment. 

• Three studies only report effect sizes in addition to the p -value of significance. While significance tests 
provide information on the probability of the sample statistics given the sample size, it does not 
evaluate the likelihood of the population of the sample. Effect sizes allow researchers to compare the 
results of their study to result of other studies (Vacha-Haase, 2001). 

• The studies investigate tool use, but the adequacy of the tools for learning or the need for students to 
consult these tools is never questioned. An evaluation of the tools themselves, and the task to be 
solved can provide evidence that these tools are indeed beneficial for learning. 

Although some indications are given of whether and how tools are used and of variables relevant to the 
issue of tool use, a firm empirical basis is clearly lacking. However, to have insight in tool use becomes of particular 
interest with the use of open learning environments. The use of these environments is advocated to foster the 
acquisition of complex problem solving or higher order reasoning skills (Jacobson & Spiro, 1995; Jonassen, 1997). In 
open learning environments, the learners construct their own knowledge in interaction with the environment. In other 
words their is a large amount of learner control and learners have to regulate their own learning (Hannafin, 1995). This 
also includes making choices towards tools to be used. To design these kind of environment it is important to know 
what variables influence tool use. 

The review presented here is of course limited. For instance, the European and Asian conference papers and 
dissertations are not considered when searching the ERIC and Psycinfo databases. More studies might be available 
on the use of tools. Another aspect relates to the subject of the review itself. Non-significant results or negative 
results are not likely to be published as positive and significant effects. Designing an environment in which support 
tools are put to the disposal of students, but students not using these tools may be viewed as a negative result. 

Conclusion 

The studies reviewed indicate different variables to influence the use of tools. Only limited studies however 
address the adequacy of tool use. 

When advocating the use of open learning environments, the issue of tool use becomes more apparent. 
Learners are left in control and decide themselves on the use of these tools. Reviews on learner control and studies 
reviewed in this contribution reveals that students hardly posses the necessary monitoring and regulation skills to 
make adequate choices, i.e. choices beneficial for their learning process. The limited number of studies addressing 
tool use in computer-based learning environments indicates that more research is needed to identify different learner, 
tool and task characteristics that affect tool use. How tool use can be encouraged also deserves additional research. 

From an instructional design perspective two alternatives to enhance tool use can be followed. One option 
would be to make the environment initially less open (reduce learner control on tool use) and provide support to 
students without students asking for it, and gradually remove this support (Clark, 1990; van Merrienboer, 1997). 
Monitoring and regulation activities are supplanted for the learner and gradually left over to the learner. In this 
option, support is not necessarily adapted to the needs of students. Some students will receive too much or not 
enough support. Both can be detrimental (Clark, 1991) as the introduction of support might increase, rather than 
reduce, the cognitive load for some students (de Jong, & van Joolingen, 1998). 

Giving students advice on the level of how and when to use the tools could be an alternative option. This 
would mean that students are left in control over the use of tools, but depending on the different steps student take, 
advice on the use of these tools is provided. This advice helps them to monitor their own learning process and to 
promote adequate tool use. An analysis of the different steps students undertake may highlight that students do not 
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posses the necessary monitoring skills. Students may then receive additional advice, while students who posses 
these skills would not receive advice. 

Some promising findings of Lee and Lehman (1991, and Grasel et al. (2000) suggest that providing additional 
advice to students can encourage the adequate use of tools. Including explanations about the aim of tools might 
result in more adequate use as well (Winne, 1985) and might prohibit students to ascribe different functionalities to 
these tools as they are intended to have. This indicates the students' instructional conceptions might be a variable 
that should be taken into account when studying tool use. 

Comparing these two options should result in insight in what approach is most beneficial to foster adequate 
tool use and support students learning process. However, research should not only focus on measuring domain 
specific knowledge, but rather focus on problem solving and whether transfer occurs. 
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Teaching High School From a Distance: Developing Effective e-Leaming Strategies 

Salley Maddox Benoit Sawyer 

Northern State University 

This case study presents a description of the instructional strategies used by the seven High School Master 
Teachers at the Center for E-learning at Northern State University in Aberdeen, South Dakota. Data for this study 
were collected through making classroom observations and conducting open-ended interviews that were tape- 
recorded. The seven Master Teachers provide daily classes daily to more than 40 school districts across the state at 
no charge to the school. The tuition-free classes include the subjects that many small schools are not able to offer 
their students, such as Physics, Chemistry, Spanish, AP Calculus and French. Each class is held every day for a 50 
minute class period in a fully two way audio / video teleconferencing environment, the Digital Dakota Network 
(DDN), and actively supported by an online component, WebCT. The system uses compressed video running over 
half a T1 line. Each receiving school is required to have a trained e-mentor in the classroom; there are no students in 
the master teacher’s studio. 

The e-learning Master Teachers are in a totally technology dependent environment. This dependency, in 
combination with their commitment to engaging the students in interaction, infuses technology into the teachers’ 
instructional strategies to varying degrees. The teachers’ expertise in teaching high school students, the depth of 
their knowledge in their subject matter areas, and their beliefs that students must be active participants in the learning 
process, contributed to the teachers flexibility with and adoption of several instructional strategies. This study will 
describe some of the strategies in the context of the teacher’s attitude toward learning and technology. 

The Center for E-learning was established at Northern State University (NSU) in the spring of 2001 and 
began offering high school courses in the fall of that year. During the summer of 2001, the teachers were given two 
weeks of training. The purpose was to give the teachers a basic level of comfort with using the technologies as well 
as to give some guidance on teaching at a distance. Most of the master teachers had no previous experience 
teaching students in an e-learning environment and many had little technology knowledge. One teacher explained, 
“the first day I sat in that training and I watched someone in front of me zipping through all these different screens to 
get to a document, I thought, ‘Ohhhh, I’ll never know how to do this.’ And here I am. I’m doing the same things 
those people are!’’ 

In addition to WebCT and DDN training by the NSU technical staff, NSU drew upon its connection with the 
Tecnologico de Monterrey in Mexico to share their expertise in virtual education. However, the teachers had little to 
no experiential knowledge of teaching in an e-learning environment and as one teacher said, “it just totally went by 
my head. They did make an effort to help us, but I didn’t know enough.’’ The teacher’s instructional strategies seem 
to vary according to their subject matter area as well as by their personal beliefs about teaching and their awareness 
of what the technology could do. 

The teacher participants: 



Kathy: French 1 and 2 


Joanna: Spanish 1 and 2 


Mary: Senior math and AP calculus 


Chuck: Chemistry 


Jeff: AP American history 


Deana: AP English 


Jackie: Physics and AP Physics 





The teachers have different approaches and opinions about the way they function in the e-learning 
environment. 

Deana, who has very small class sizes, opens each class by recognizing the school and the individual 
students. She often engages students in brief personal conversations about their school and sports activities. 

So they know that Tm looking at their classroom ... I need their input all the time ... I try to touch base with 
them a little bit personally. Because I think if I know my students a little bit personally then they instruct 
easier. 
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Kathy takes delight in planning learning activities that are truly active. She constantly seeks ways that will require 
students to apply their newly developing knowledge 

And I want my classroom to be interactive, hopefully we will have some enjoyment and some days I feel like 
that’s why I came this year. And it’s something kids look forward to coming to. You have to realize that 
seven schools in the morning show up an hour before school starts in their town, to take a class. And that 
you have to appreciate the fact that they’re willing to do that. And that at 8:30 I have kids west river who 
are actually there an hour before their school starts. And I have one group of eleven in the afternoon who 
skip lunch to take French. So these kids are making sacrifices in order to take that class and I think that 
needs to be considered too. So when I say there should be something there for them, there should be some 
form of enjoyment there for them. 

Mary emphasizes the visual nature of the e-learning environment. She uses the overhead document camera, the 
Elmo, in every class and frames her camera shots to convey information rather than “telling” information. 

it has to be visual first and then everything I use outside of that compliments the visual. But if it isn’t visual 
first. I’m probably not going to make it. It may fail as a strategy if isn’t visual first. I can use lots of other 
things like I can be enthusiastic when I talk about things, I can let kids talk about things, I can do lots of 
other adjustments, but if I can’t make it visual first. I’m probably going to have trouble. 

Jeff has began teaching AP history this year rather than American Government. He has begun using PowerPoints in 
his class as a way to present visual information. 

One thing that I do a lot more of now, especially with my AP history class, that I didn’t do before when I 
taught in a regular classroom ... I use PowerPoint a lot more because it’s so conducive for this environment 
with the TV and the computers being right there. And so I do a lot of PowerPoints getting up paintings or 
pictures of people that I’m talking about so that they can see who I’m talking about while I’m talking about 
them. 

Jacki is new this year to the e-learning environment. She has been conscious of the time delays in the DON system 
and seeing things from a students view. 

First of all PowerPoint helps me to remain more focused on what I need to do but it also takes a lot more 
work ... I did a lot of drawings and stuff on the blackboard. Now I find a do more of my drawings in 
PowerPoint . . when I started making my PowerPoints, I had no idea the room to use and space as far as 
within the constraints of the computer. So then I put some of my PowerPoint presentations up in one room 
and then go into the other room and look and see what it looked like. So I’ve had to be more conscious 
about what it’s looking like at the other end. Where in the classroom, you just did it. 

Joanna is frustrated by not finding ways to learn more about teaching in an e-leaming environment. 

the techie part of it, the working the V-Tel tablet, all that stuff I’m comfortable with. I think we all are. For 
me, just from my perspective, it’s the strategies of communicating, of getting the kids to interact with one 
another across the distance, not those within their own schools but across the distance. Getting them to 
interact with me more rather than sitting back and not wanting to say much. That’s what I struggle with 
most. 

Chuck discussed his strategies and underlying philosophy as being the same in the f2f and in the e-leaming 
environment. 

students have to move from concrete to abstract. And to do that I used what’s called constructivism 
theory. That just means that I want my students active and I want a lot of hands on. I want them to be able 
to manipulate laboratory equipment and materials before we talk about the explanations and the theories and 
the elaborations of the concepts . . . So, when I’m doing demonstrations I want to make sure that they see 
clearly and that the time is allotted so that they can read instrumentation even though they can’t put their 
hands on it, they can put their eyes on it 

The DON and WebCT technologies are not part of a course delivery systems; they are new learning environments 
that require conscious effort on the part of the students and the teachers. Some of the teachers have strategies for 
guiding the students with both the tools and interactions in the e-learning environment. Deana explained, 
“Sometimes I will just put up some directions or “how-to’s” on the Elmo. I have done it where I have taken my 
computer in and I have told them now this is how to get to my home page.” Kathy uses activity-based strategies 
designed to facilitate interaction between students in different schools. 

I have people work through the system if they’re isolated. For example I have one student in Redfield, one 
student in Canestota, one student in Bowdle. When I assign group work will ask the other people to mute 
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their mikes. While they practice their conversation and those three will practice a conversation through the 
system. So they get the same experience of talking to students as everybody who has a pair or more in the 
classroom. So that’s a strategy I’ve used a lot because I do have isolated students. Which is difficult by 
the way. It makes it very hard to do group work. I have even gone so far as to have kids write skits and e- 
mail them to each other, the conversations. One person will volunteer to type them up and e-mail it so they 
can do the conversation from three different sites. 

The teachers have developed strategies that combine instructional management with facilitating interaction. Both 
Kathy and Mary use the strategy of writing each student’s name on a 3X5 card. The cards help the teachers to 
function in the environment and they help the students to stay engaged. Kathy explained, 

When you’re dealing with seven districts, it would be easy not to call on somebody because they’re not all 
in front of you. I can only see one of them at once. So I go through those cards religiously. I call a name, I 
set is aside, I call a name I set it aside. I remix them so they come up differently, but I make sure that every 
student is called on an equal number of times in my classroom. I think that’s part of a strategy and it’s a 
strategy that works. 

For Mary, the cards simplify her tasks when teaching. 

Just when I put the problem up on the document camera . . . and then I say who is going to graph and the 
kids’ names are all on cards in my folders . . . and I just lay that card up. And again that’s just a survival 
tool. I’m running too many things. I never did cards ever in my life. I do cards all the time now. And if 
there’s some little thing I want to remember to tell a kid or that they’ve told me in e-mails or something, I jot 
it on the card so I won’t forget it later on. 

Planning time in the e-learning environment has been an unexpected change for all. The teachers must manage the e- 
mentors, prepare material kits for several districts, select the camera shots during each digitally recorded class and 
constantly update the WebCT sites. Joanna observed that, “When I took this job I thought I probably wouldn’t 
spend as much time as I did in being physically in the class, but I spend more. There’s no question in my mind that I 
spend more.” Time consuming, as it is, all the teachers described the importance of planning in the e-leaming 
environment. Kathy stated, “When you talk about strategies, number one you better talk about your planning.” 
Chuck remarked about the time it takes to plan chemistry experiments in ways that will ensure problem-based 
learning. 

I’ve taught this basic chemistry class for 25 years previous to last year when I first taught it in e-leaming. I 
didn’t think I would have hardly any planning, I just felt it would be spontaneous as it has been for years 
and years. But every detail has to be in place. It’s a production. It’s no longer just a teaching situation. 
Using WebCT becomes part of everyone’s instructional strategies. The teachers use WebCT for communicating 
with students as well as the e-mentors. Chuck explains. 

It’s very challenging to get high school students to use it well enough. Of course that’s a developing thing 
with me, I still am developing a better use of it also. . . . The biggest thing I like about it, the strategy that 
works pretty well with me is the communication tool. Because I require my students to do all of their 
communications to me through that tool. It’s just a more efficient way to keep track of who’s 
communicating what and I can get back and I can keep it in an electronic file. 

For Joanna, WebCT has become a time saver. 

I wouldn’t be able to live without it now, after using it. It’s like a focal point I can say download this 
document for tomorrow instead of ok I’ll fax all of you this document or I’ll mail all of you. It’s just a lot 
easier. It takes time to get ready, but without it I don’t know what I would do, it would be more of the old 
way of teaching 

Mary has created a WebCT page to introduce each section in the math text book to tell the students about the main 
points. She has made sure that her writing has never exceeded one screen and that the writing style is very 
conversation, “plain ordinary English.” She also uses the calendar tool. 

it’s absolutely wonderful. I like the calendar; I update it every Friday. Teachers in schools usually have to 
hand in lesson plans to their principals on Friday. I told the kids that they were my principals and I have to 
have the next week up on Friday. And they expect it and I expect it to be there 
Both Chuck and Mary attributed the e-learning environment at NSU as providing an ability to develop and refine 
instructional strategies. Mary reflected that her teaching improved because she is not pulled away for more 
traditional f2f tasks as bus duty or lunchroom monitoring. This year. Chuck has refined his use of e-learning 
technology as he travels from the NSU teaching studio to visit the high schools classes. As he learned how to edit 
his digitized classes the visits have fed back into his strategies. 
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the other strategy that I’ve really changed this year is what I tape of my classes and what is taped from my 
studio. Now last year almost exclusively just because I hadn’t branched into the area of actually being able 
to record more from the other sites, as well as from my own studio. So I have, my class clips are much more 
exciting. Why is that? Because we see more of the kids. It’s the students that are on my class clips and 
their responses and their work and so I’ve relinquished a lot of time from me to them, which is a good 
strategy 

Mary described the e-learning environment in combination with the tech support as giving more time to concentrate 
on teaching. 

I think environments are different and this is a very supportive environment that encourages you to do the 
very best. Yesterday I wrote a test for my AP calc kids and it’s a test that I’ve dreamed for a million times in 
my life and never written because I’ve never had time to do it. And that was where I took the pieces from 
the first tests and added C’s, D’s, E’s and F’s that let them take what they started with in the very first 
chapter and apply it to what we are right now. So they could see how the whole thing, it was a progression. 
It all linked together. We first started with limits then we went to continuity and then we went to derivatives 
and they all work together. So my problems, and there were only four of them on the test, went from limits to 
discussing continuity and then to discussing the derivative and then to using the derivative to write the 
equation of a tangent line. It was all from a problem that we started before and I’ve just never had time to do 
that before. 

Along with appreciating the opportunities that the e-learning technologies have made available to their students, 
some of the teachers have come to see the technology as enhancing their teaching strategies. Chuck explained. 

Just for example, having the capability of interfacing my computer with a screen through a projector or a 
monitor, I can actually just solve problems, I can demonstrate problem solutions, and then I make the same 
kind of mental errors students make and have to correct those and it’s just really a neat way of interacting . . 

. I have access to three camera sources, I can play DVDS, CDs, LP disks, I just have access to every 
possible kind of imagery there is right now . . . again, without some technology I just can’t imagine the 
modern classroom functioning. It’s just such a powerful support to teaching, any teaching strategy can be 
supported with technology. 

Kathy sees the technology as providing opportunities that will let her students see and interact with a world 
something outside South Dakota. 

I think one of the things that the distance classrooms provide for students that they did a little of in the 
schools that is wonderful is the way they learned to use the WebCT and technology . . . there is a worksheet 
on Paris where they go to three sites. I mean there’s opportunities now that have never been there before . . 

I have an interactive weather map of France where they can go to any city in France and see what the 
weather is going to be for five days. And I took the vocabulary from that and made a vocabulary sheet for 
my second year students, of actual weather that’s posted on the internet. You know, rather than the “il fait 
beau” and the things you learn in first year it gives them a little depth. So that if they actually looked at a 
weather map they’d know what they were seeing. 

While appreciating the technologies, each of the teachers remarked about the things they missed having in the e- 
learning environment. Jeff missed the out of class contact with the students. He missed seeing students and their 
families at sports activities or seeing them at the grocery store. Deana missed being able to walk around the 
classroom and look over a student’s shoulder. Joanna was troubled by the audio and video delay and missed the 
instantaneousness of a f2f classroom. Jackie misses being able to see all the students at the same time. Chuck 
missed being able to do the “real chemistry” that he could in a lab setting with students. Yet each of the teachers 
appreciated “specialness” of what they were doing; they were reaching students who would not otherwise have 
access to this kind of education. 
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Best Practices Congress: Stimulating Faculty Interest in Online Instruction 

Paul B. McKimmy 

University of Hawaii- Manoa 
Abstract 

In Fall 2001, Texas A&M University-Corpus Christi invited faculty to participate in a Best Practices Congress 
focused on online instruction. Faculty members that were teaching through web-based course tools were invited 
to demonstrate their courses to those who were exploring the idea of online instruction. After a structured “show 
and tell” period, a discussion was facilitated focusing on pedagogical issues. Participants indicated that seeing 
their colleagues’ strategies and approaches to online instruction helped in planning their own courses. Strategies 
and best practices were compiled from the pedagogy discussion and distributed to participants for reference. 
Strategies included use of practice tests, photo rosters, assignment benchmarks, games, course logos, and third 
party supplements. 

Best Practices Congress: Stimulating Faculty Interest in Online Instmction 

Teaching using online can be a consuming experience for new users. After tackling the initial learning 
curve; understanding the available tools; experimenting with design activities; troubleshooting technical problems; 
and converting existing content for online use, it can be difficult to think about new instructional approaches. 

Sharing ideas and novel approaches can be an important part of improving online instruction at your institution. For 
this reason, Texas A&M University-Corpus Christi held its first Best Practices Congress to facilitate the 
dissemination of online instructional strategies. 

In Fall 2001, Texas A&M University-Corpus Christi implemented Web Course Tools, WebCT, on a trial 
basis for a small group of faculty. Faculty used WebCT to implement a variety of web-enhanced and web-based 
courses. The “Best Practices Congress’’ was assembled to share success stories and instructional strategies. 

Faculty members with WebCT accounts were invited to attend and to present an idea that they had implemented 
through WebCT. The entire faculty was invited to view the informal presentations and to learn about online 
instruction. 

Two sessions were held on separate days to increase faculty availability and participation. Twenty-two 
faculty members participated in the sessions. Each session began with an introduction of faculty, an overview of the 
discussion format, and live demonstrations of individual courses as designed by participating faculty members. 

After circulating informally to see each course design and ask questions of course designers, discussion ensued. 
Faculty members were asked to discuss implementation ideas as they related to specific pedagogical issues 
including: content organization, interactivity, multiple modes of delivery, assessment, learner involvement, technical 
issues, games, and simulations. 

Participants indicated that viewing their colleagues’ strategies and approaches to online instruction helped 
in planning their own courses. Participants found several ideas to strengthen their own use of course management 
tools for Internet enhanced instruction. Strategies were compiled from the pedagogy discussion and distributed to 
participants for reference. Those strategies are reviewed here, including the Best Practices Congress itself. The 
following ideas were presented as some of the most useful approaches to leveraging WebCT and related tools for 
instructional strategy. 

A strategy proposed for assessment and involvement was the use of practice tests . The instructor creates 
sample tests for students to attempt when they believe themselves ready for the real exams. These shorter tests 
include feedback on incorrect as well as correct answers, and are required as part of the course participation grade. 
Automated feedback provides a degree of interactivity to this teaching tool. Using conditional release, the instructor 
times the practice tests to be available at an appropriate time and to disappear after the actual exam. This forces 
students to earn their participation points when the practice tests will help them. If the practice exam availability is 
not limited, students with poor time management skills might leave them unused until late in the semester. (Idea 
contributed by multiple sources) 

Another strategy, use of a photo roster, encourages student-to-student and student-to-instructor 
interaction. For a hybrid format course, instructors use a digital camera to capture student images. Students create 
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table tent nametags for themselves , using large letters . The instructor photographs each student with his or her table 
tent. As an alternative, the instructor can add names using Photoshop or other software instead of using nametags. 
Using these photos, a simple table is created in any html editing program (Claris Homepage, Netscape Composer, 
even Microsoft Word will do). Photos are inserted into each cell in the table and the page is uploaded to WebCT and 
added to the course homepage. This photo roster assists the instructor in learning student names quickly, and 
assists students in learning their peers’ names as well. The entire process facilitates ease of discussion in 
subsequent meetings by allowing the instructor to learn and use student names quickly. (Idea contributed by Dr. 

Paul McKimmy ) 

Using Assignment Benchmarks, instructors are able to model quality work to students. Instructors seek 
permission from former students to their work on the course page as examples of strong essays, projects, or 
presentations. Other students will be able to quickly form an idea of the expectations involved in receiving high 
marks or developing a strong presentation. (Contribution source unknown) 

The Strongest Link is an ingenious combination of TV game show themes with a positive spin. Using the 
game framework, the WebCT discussion tool is employed to develop writing skills, critical thinking, and even reduce 
the instructor’s workload. This activity can replace some class periods due to the research and response time 
required. The instructor posts brief case studies that students must research and write responses. (Example: In 
International Accounting, a question might describe the financial War on Terrorism, asking how one could audit the 
accounts of suspect charities to determine which monies might be funding terrorist activities) After the responses 
are posted, students vote on the “strongest link” - the best response. The student who submitted the best response 
is then excused from submissions for the remainder of the semester! Gradually, the instmctor works with fewer 
students and can concentrate on those that are missing the boat. (Idea contributed by Dr. Valrie Chambers) 

With permission, video clips, outlines and presentations from 3'“* parties can make excellent supplemental 
material. In many cases, textbook publishers provide PowerPoint slides, chapter outlines, and other resources with 
instructor materials. These files can be uploaded to the course and used by students for review. As an example, Mr. 
David Kendrick uses a video from the publisher of his Microcomputers in Education text to supplement his course. 
Email was used to obtain owner permission and Media Cleaner Pro was used to reduce the file size sufficiently for 
Web use. Students were able to download and view this video directly from WebCT. One of my students requested 
use of my lecture outlines, which I uploaded to the course page. I later observed the student studying from my 
outlines at our fitness center. He subsequently improved his test scores. Conditional release can be used to delay 
use of similar material until after class discussion in order to encourage students to use the textbook rather then 
depending on supplemental outlines. (Ideas contributed by Mr. David Kendrick and Dr. Paul McKimmy ) 

Many instructors also created a banner or logo for their courses. The banner was included on all the 
primary pages within the course. Those instructors who designed a graphic or “banner” for their course noted that it 
added a professional image to the course. Adobe Photoshop or another graphic editing program could be employed 
to create the log. (Contribution from multiple sources). 

Perhaps the most intriguing idea for using course management tools for improving instruction was the Best 
Practices Congress itself After participation, faculty indicated an increased interest in using course management 
tools and an increased understanding of available resources. Eurthermore, they were able to identify other faculty 
members who were using course management tools effectively and who could act as peer support for new designers. 
The Best Practices Congress is recommended as a method to stimulate broader interest in online instruction. 
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How do Educators in High Schools Accept the Internet as a Learning Medium 

Avi Cohen 

Abstract 

The goal of the present study clarifies the viewpoints of teachers in Israeli high schools as to how they 
might accept the Internet as a learning medium and education purposes. 

Research Method was qualitative and the main tools were interviews, analysis of sites and documents. 
Twenty-two in-depth interviews and thirty-two random documented interviews. The in-depth interviews were 
analyzed by three analysis levels: location of categories and criteria as is accepted in the “constant comparison’’ 
method, episode analysis and analysis of the teachers’ full stories. 

The research deals with three perspectives: External to the school, curricular and pedagogic aspects within 
the school and the teacher’s personal intrinsic status. 

The main findings of this research show that the Internet serves as an external cultural motivator, which 
influences parallel internal school processes. A distinction should be made between distance learning via the Internet 
and education - it seems that high schools are mostly educational/social institutions. The traditional school structure 
and the teacher’s central role in the educational process should be maintained - the teacher is in the heart of the 
socialization process. The Internet can influence teaching methods and by doing so can turn the teacher into a 
pedagogue rather than a teaching technician. The education system is responsible for the means of transferring 
information rather than for the supply of end-technologies. It seems that the correct phrase for high school is 
Distributed Learning rather than Distance Learning. Distributed Learning on the Internet pushes education and 
learning fromset years of education into Life Long Learning and places the teacher as a leader of this change. The 
full version of the research findings are placed in http://www.AviCohen.info 

Introduction 

We have lately become aware of the promises and challenges that the Internet environment sets for teachers 
and of the changes they are required to make. The present study in this article focuses on the way teachers regard 
the changes involved in introducing the Internet into schools and how they view this medium as an educational and 
work tool within the didactic and curricular framework of their high school. The intention is to inspect the viewpoints 
of “regular’’ teachers that engage in traditional learning, and to examine how they cope and identify with the change 
processes that the Internet brings to the school culture and ethos. The question is asked how a teacher’s perception 
of his role towards himself, his students, society, the education system and the inner-school restraints can use the 
cyber infrastructure as an additional teaching medium or as a replacement of existing teaching methods. The 
conceptual framework is supported by three components of school teaching. The first is change processes, the 
second is how teachers relate to and organize teaching and knowledge, and the third is maturity processes and the 
attitude to the Internet in the near and distant environments of the school and mainly the Israeli Ministry of 
Education. 

The rate of penetration of computers and other digital information means in Israel is as rapid as in other 
advanced countries. The rate at which schools are connecting to the Internet is steadily and quickly rising. 

Melamed et al (1999), quote impressive numbers of schools that are hooked up and the rate of increase and 
state that the penetration of computers to all walks of the economy is comprehensive and deeper. The authors laid 
the foundations for speeding up the computerization process in the Israeli Department of Education. The objectives 
set by the policy makers relate to the penetration of information technology and progress of pedagogic goals by 
means of the Internet at all levels of the education system. The means to creating the conditions necessary for 
implementation of the goals and objectives are: physical infrastructures, accessibility to computers, teacher training, 
material preparation, development of a national digital library, promotion of pedagogic models and their distribution. 
Melamed et al (1999), also claims that the introduction of computerized learning processes requires a change in the 
school culture, adaptation of the learning environment to the new technologies, adaptation of the pedagogic 
processes to the new learning environment, training and support of the teaching staff, and adjustment of the learning 
materials to the new processes and environment. The authors assert that there is a need to establish a network of 
3000 schools and training centers to the Internet, on a line that will enable simultaneous operation of at least 20 
computers. The net should enable connection of every teacher and every student from anywhere, Internet provider 
cost-free and at a reduced fee. 
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There are a number of public initiatives in Israel concerned with the development of virtual high schools. 
These initiatives take place within the framework of the “Aviv” School, the Hebrew University and the “Snunit” site. 
The Center for Educational Technology is conducting an experiment within the formal education system, in which 
teachers give fully virtual courses in a number of schools. Appendix 1 1 includes the abstract of a lecture and 
discussion at the Center for Educational Technology concerning innovative projects of distance learning. The 
opening sentence of the lecture expresses the need of introducing the Internet and distance-learning infrastructure 
into the education system: 

“What we defined for the next few years relies strongly on virtual teaching... The education system is 
trying to see what can be gotten out of the newly emerging technologies... We think that the education 
system should try to discover the points that it can harness for the purpose of computer development... 

The first enterprise is that of the teachers' communities... ( Cohen, 2000, p. 408) 

The Internet as a medium that is growing and taking an important role in educational activity has galvanized 
the Ministry of Education to action. In March 2000 the Science and Technology Division established a forum called 
the “Content Committee” the task of which it is to generally define guidelines of qualitative standards and evaluation 
criteria of some of the information on the Internet. The operative objective is to produce a “statement of position” 
that will provide goals for the consolidation of a comprehensive Department of Education policy. Division activity 
has permeated the schools, and the opinions regarding teaching methods and distance learning swing both ways. 

Salant (1999), is aware of very rapid progress in the US, UK and Europe The American effort is now 
changing track from technologic connections of schools to the Internet to two new directions that include training 
teachers and adaptation of digital content. There is some pressure of community and business circles to increase 
Internet applications and integrate them in the schools. Salant (ibid) adds that the main problem facing the American 
education system is the training of teachers for Internet applications. An interesting point, which will be discussed 
extensively in the findings chapter of this study, is that contrary to expectations, novice teachers do not use the 
Internet more than veteran teachers. 

Cuban (1999), states in the third annual report of Information Technology Applications in Education that the 
US has a growing problem of Internet sites compatibility with the existing learning programs. The American 
Department of Education is active establishing a rich and indexed database of learning materials. This project, known 
as EREE, is a federal initiative supported by Congress (US Department of Education, 2000). An example of some other 
interesting projects operated in the US is the Michigan project (Michigan, 2000). 

In Britain, the rise of the Labor government in 1997 brought about a national change in the public issues, 
and Britain’s main investments today are in education. Within this framework, the connection of British schools to 
the Internet has become a national priority. NGEL (National Grid Eor Learning) is the national British project for 
computerizing the education system. The Prime Minister, Tony Blair, initiated it in 1998. The NGEL project consists of 
three main stages: connecting schools to the Internet, establishing education resources centers for teachers (VTC), 
and developing education materials and databases for teachers and students. Despite the British national effort, most 
of British teachers still do not use the Internet for teaching purposes and do not integrate it in the classrooms, for 
two main reasons. Eirst, the teachers tend to cling to a centralized curriculum, and second - relatively low 
investments so far in training and pedagogic projects for teachers (UK Education Department Site, 2000). We find 
similar policies in Germany, Erance and the Scandinavian countries (Salant, 1999). This Artical deliberately refrains 
from discussing budget details. My intention is to present trends and processes that produced the conceptual 
framework of the research study. 

This research has a wide framework, which stems from change processes; from the way teachers regard their 
teaching, from how teachers act and from the entrance of the Internet to the education system. I will try to examine 
the teachers’ statements and their specific attitudes to the Internet as part of the learning process at school within 
this framework. 



Methodology 

Research Method was qualitative and the main tools were interviews, analysis of sites and documents. The 
study included an exploratory study and a main study. The main study, as well as the exploratory one, was 
accompanied by examination of documents and material gathered from various Internet sites, which serve teachers 
for teaching purposes. Twenty-two in-depth interviews, thirty-two random documented interviews and sixty other 
random not transcribed interviews were conducted. The in-depth interviews were analyzed by three analysis levels: 
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location of categories and criteria as is accepted in the “constant comparison” method, episode analysis and analysis 
of the teachers’ full stories. 

The initial category of the interviewee’s professional specialization served as the first sorting category. 
Common professional specialization was selected: languages, sciences, fine arts, social studies, technology and 
computer science. Another selection was made by role at school, age (young, middle and old), gender, and type of 
school (academic, technological or religious schools). The interviewees were selected so that each was experienced 
in Internet communication. 

The planned interviews were ethnographic interviews, and guided and focused interviews were the main 
research tools. The interviews were later taped and transcribed in the method known as “interpreted interview”. The 
interviews lasted between two and four hours. Most lasted about three hours. Random interviews were also 
conducted, and most of the random interviews were unplanned. They occurred whenever the researcher encountered 
a teacher or an education official, who was able to shed light on an aspect of the study or to validate what had been 
said. The average duration of a random interview was half an hour. The longest took an hour and a half. The random 
interviews are a purifying and enhancing process for the categories presented in the findings. These interviews 
supply and support the generalization and validation capabilities of this research. During some of the interviews, and 
visits to educational institutions and the Ministry of Education, documents were received that were in some way 
related to teaching through the Internet, analysis regarding this matter, teachers’ work -papers, and Internet sites 
created by teachers while working with their students. This material served to validate the participants’ views, to 
construct the categories and to present the results and conclusions of this study. 

This study made use of a number, or rather a combination of analytic approaches with content analysis 
methods befitting the nature of this study. The most suitable to the present study is “analytic induction”. 

Three analysis levels were used to analyze the findings. On the first level of analysis a “statement” was 
chosen as the analysis unit. A statement is defined as the interviewee’s words on an issue from beginning to end. In 
order to develop a “ground theory” the analysis created categories and criteria for their definition. The first step is 
“open” coding, which means locating repetitions and their characterization through categories (coding level I). The 
next step is “axial” coding based on properties located by means of open coding (coding level II). The final step is 
“selective” coding in which the material is coded by pre -designed categories that were located in the previous steps 
(coding level III). Data analysis began at the very first stages of data collection. The coding steps went on in parallel 
throughout the data-collection process. This method allows for refinement and redefinition of the observations made 
at the initial stages, and prevents the researcher’s bias, which could turn permanent properties into interview 
questions or points to look for in documents and observations. The second level of analysis was the “episode” 
analysis unit. An episode is an occurrence of independent interest and content, the beginning and end of which are 
marked during the interview. This unit was defined as “the teacher’s account of an event or occurrence or way of 
teaching including background, results and related feelings”. The episodes were divided into groups by free 
connection to some of the categories found at the first analysis level. Random interviews, which relay episodes, 
played a major part. The interviewees each in their time and place did not speak about all the topics, but only of those 
they thought of and the situations they encountered. Their communications were very powerful and added a certain 
flavor to the atmosphere in which the teachers find themselves when they enter the classroom and carry the burden 
of the cultural environment outside the classroom. On the third level the analysis unit was a full interview with one 
teacher. An attempt was made to connect the teachers’ background with how they regard teaching and Internet 
technology as an additional layer of their own teaching and teaching in general, as reflected in their communications. 

An effort was made to enhance credibility and validity in some ways. Responsiveness, fairness, 
trustworthiness and accuracy were used. It seems that interviewing is an appropriate tool to ensure inner validity. 
Comparison of the various teachers’ accounts (by “cross-reference”) and reference only to findings that were 
frequently repeated served this objective. Additional validation was achieved by the “tripling” method, such as 
document analysis. The “documents” in this work were primarily Internet sites, which the teachers referred to and 
which were examined as result of this and of other publications. It seems that selecting a sample of 22 in-depth 
interviews and 32 random documented interviews and dozens of undocumented interviews distributed by the main 
categories of the Israeli education system, and examining accounts of other teachers as reported in the interviews - is 
sufficient to ensure external validity without the power of sampling error in quantitative researches. There were 
adequate repetitions of the teachers’ perceptions and beliefs, so one can assume that most of what they report did 
indeed happen. It was therefore possible to reach conclusions about how teachers act when they are given an 
additional tool. A more complex problem concerning external validity is whether the conclusions are also valid for 
teachers in Israel and worldwide who are not exposed or accessible to Internet technology. Apparently for some the 
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answer is yes. The detailed presentation of findings by a method of “thick description” enable to discover 
conclusions valid for other teachers as well, by means of comparison and contrast. The entry level of a subject into 
the sample population is low and general enough to include most teachers in the category. At the conclusion of the 
study it is apparent that with time, more and more teachers can become part of the appropriate population. 

Analysis validity was performed in two stages. First, 2 Judges who are specialists in analysis of qualitative 
research methods were requested to analyze two interviews each, to identify the validity of the categories located by 
the researcher. At the second stage the same two judges were asked to use the previously formulated “properties” 
system to analyze two interview texts. The properties system was duly corrected following the judges’ remarks at 
both validation stages. The initial correlation was nearly complete. Structure validity was enabled by seeking the 
assistance of readers, some of who were not teachers, to validate the final conclusions and the general picture that 
evolved from the analysis. There was almost full agreement about the conclusions and the theoretical model 
presented. 

All participants were assured full anonymity, to prevent their own or their school’s identification. The 
researcher alone kept any personal details. All names mentioned in the work are fictitious and other details were made 
general so as to avoid identification. This item was fully examined with the assistance of the readers. Full consent of 
all participants was ensured, as well as the prerogative to withdraw at any stage. 

Main findings and discussion 

The first perspective of the findings in this article is external to the school. This perspective deals with the 
influence within a global social framework and the teachers’ relation to the school’s external interested parties. The 
main findings that deals with the external perspective are: 

1. The Teachers’ Perception of the Cyber World as a New Culture - The teachers {namely the 
interviewees in this research) view the Internet as a new culture. The topics that characterize culture 
are: language, democracy, and a way of life, relevance and common interests. The degrees of relevance 
and accessibility presented by the teachers constitute the enabling environment of this culture with all 
its advantages and disadvantages. 

2. The Teachers’ Perception of the School Struggle to be Relevant as a Result of Internet Assimilation 
Into Everyday Existence - The new culture is not threatening and should be allowed in the schools. 

The teachers feel that at times the pedagogic consideration is not the only one. Principals and 
headmasters take advantage of the Internet process for populist purposes and as a recruiting tool. The 
teachers view the procedure as a battle for the students’ attention. 

3. Traditional Teaching in High Schools Versus Distance Learning Possibilities in Higher Education - In 
high school, traditional teaching should continue with the Internet. The teachers stated that high 
schools and universities have different attitude. They expressed the need for traditional learning in high 
school as a social need, youthful exuberance that must be controlled, responsibility, maturity and 
motivation. Those that rejected distance learning in universities did so for reasons of socialization and 
isolation. Most of the teachers displayed dissatisfaction with the pressures, the problems and the 
conflicts of the new culture and the social needs of high school. 

4. The Teachers’ Perception of the Government’s Role in Supporting and Sustaining the Process - The 
teachers stated that the government and the Ministry of Education must take responsibility for Internet 
penetration into schools. Three integrated areas were diagnosed: pedagogic, organizational and social 
The teachers claim that the government should deal with curriculum, staff, hardware and software, help 
to the weak and training. They expect the Ministry of Education to supply pedagogic and 
organizational management, and believe that Internet assimilation should become a national project. 

The teachers lay some of the responsibility at the parents’ door. 

5. Perception of the Internet by the Ultra -Orthodox as a Tool of Agitation - The orthodox education 
system in Israel rejects the Internet by a number of degrees. The severest level is total and unqualified 
rejection. But, cross-reference and further examination of the orthodox teachers’ statement revealed 
internal differences attributed to various rabbinical followings. Some orthodox communities allow the 
Internet for livelihood purposes. Others allow selected sites with special control and censorship 
mechanisms. The central objection to the Internet is a moral one. Many religious educationalists avert 
their eyes from the rabbinical decrees 
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The second perspective of the findings in this article is aspects of Internet integration in schools from a 
curricular-pedagogic viewpoint, while examining the limitations and relationships with internal interested parties 
within the realm of the teacher’s job and responsibilities at school. The main findings of the inner perspective are: 

1. Methods of integrating the Internet in teaching as a “giant library” or a “sea of information” - The 
characteristics of the information sea are focus, orientation and validity . There is no clear decision 
between interviewees who regard the textbook with nostalgia and see it as solving the three problems 
created by the characteristics of the “information sea’’ and those that view the Internet as a new 
replacement to the irrelevance, limited number of pages and the non-updated state of traditional 
textbooks. 

2. Wavs of Combining Asvnchronic Communication and Excluding Synchronic Communication in the 
Formal Teaching Alignment - It seems that the teachers side with asynchronic communication (i.e. e- 
mail and forums) and oppose synchronic communication (chats). The arguments are usually 
seriousness versus the shallowness created by chats. 

3. The advantage of incorporating subjects that require long intervals of interaction over subjects that 
require short interaction intervals and the advantage of English teaching - The teachers divided their 
attitude to teaching via the Internet into three categories: humanistic and social subjects, technological, 
realistic and engineering subjects and languages. The humanistic and social subjects are superior both 
in synchronous and asynchronous learning. The advantages of teaching English by means of any 
technique offered by the Internet are clear. 

4. The preference of using the Internet in writing study-papers as an addition to traditional examinations - 
The teachers find that the legitimate mode of evaluation is an examination or a paper. The teachers 
opposed traditional examinations via the Internet, but were considerably in favor of papers that use the 
Internet as an addition to traditional exams. The main argument was lack of trust in the pupils’ integrity 
and honesty. On the other hand, there was no problem concerning lack of trust when it came to study- 
papers. 

5. The rejection of acknowledgement of pupils who studied by means of distance learning in high school 
- It seems that teachers neither trust nor legitimize high school pupils who studied via distance 
learning, and would want to examine the quality of the learning by a traditional synchronous exam. The 
teachers’ statements were full of contradictions. At the start they expressed support of distance 
learning and at the end mistrusted the acquired knowledge. Nevertheless, the teachers seem to give 
more legitimization to distance learning performed by acknowledged and physically real institutions 
(with walls). 

6. The incorporation of Internet censorship in school versus its rejection in the pupils’ homes - The 
teachers regard sex as the Internet’s main evil. Accessibility enables entry to problematic sites while 
preventing censorship. Most interviewees were in favor of (politic) censorship of sex sites in schools, 
but against it in the pupils’ homes. The ultra-orthodox block the Internet due to this evil. Some 
interviewees raised issues of anti-humane sites or copyright, but were mostly against censorship due to 
these reasons. 

7. The integration of the Internet as an informal and casual communication channel with pupils and 
colleagues - The teachers pointed to Internet sites as possible forms of notice-boards about learning 
procedures, messages concerning meetings or cancellations, and of placing procedural responsibilities 
with the pupils. Communication of messages through the Internet would work only if the regular 
information channels were not functioning. Some of the problem of strengthening the administrative 
and pedagogic ties is due to an assimilation problem that also exists with the pupils. 

The teachers spoke of the school and the internal interested parties on two main levels: the pedagogic level , 
which is concerned with ways to integrate the Internet in learning and learning control, and the organizational level , 
which is concerned with the transfer of messages between teachers and pupils and the organization of learning. The 
two levels are connected and intertwined, which at times creates accord and at times - conflict. 

The third perspective discusses how the teacher views himself - how he sees himself in a changing world, his status 
versus his pupils and society as a whole, his position as a leader of processes, the characteristics of camp leaders, and the fears of 
teachers in their contact with the Internet. The main findings of the intrinsic perspective are: 

I. The teachers’ perception of themselves as pedagogues and leaders of learning as opposed to the 
ignorance and incompetence that might be perceived by the students - The status change of teachers 
as a result of the introduction of the Internet in schools involves two contradictory processes. The first 
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is how the teacher perceives the role. Role perception is manifested by how the teacher sees himself 
versus his pupils, personal development opportunities, and his relationships with his colleagues. The 
second process of change is the teacher’s perception of the role in the pupils’ eyes. That is to say, how 
the teacher thinks the students will see him and how they will behave following his change of status. 
The first process is perceived as joyful, creative and optimistic. The second process is frightening and 
causes aversion, and the interviewees sought a way to justify their difficulties. 

2. The teachers’ perception of their role as irreplaceable educators - All interviewees without exception 
expressed adamant opinions about the teacher’s role in the classroom. It is the interviewee’s belief that 
the high school teacher is irreplaceable. They want to see their pupils’ eyes and to feel their proximity. 
This finding may have long-range significance in that the interviewees’ standpoint is not dependent on 
technology. 

3. The teachers’ perception of technical expertise combined with unexplained phobia as an obstacle to 
using the Internet - Two strata of technical computer problems influence the teachers. One stratum 
concerns technical problems of uncomfortable interfaces (familiarity with software, switches, etc.), 
operating expertise (familiarity with Internet options) and technical problems (non-functioning computer 
or printer). It appears that good training would solve these problems. The second stratum pertains to 
psychological fear (one interviewee called it “the wall’’). It is quite likely that the first stratum enhances 
the second. 

4. Perception of the Internet as a media that requires additional time and effort during the teaching process 
- There is no doubt, all interviewees think that the Internet does not save effort. They feel that they are 
not rewarded for their additional endeavors. We found two opinions regarding effort and reward. One is 
frustration that cannot see the light at the end of the tunnel. That is, they do not foresee that the 
system will appreciate their hard work in the future. The second is optimism that is sure the efforts will 
pay off. It seems that the teacher considers himself as fulfilling a mission. The teacher sees his work as 
thankless and his success as a mission. 

5. There is no characterization of teachers who lead the process of introducing the Internet in schools - 
The interviewees’ statements indicate that entering the subject is dependent upon the teacher’s 
personality, his motivation, the school the teacher teaches in, the type of pupils, available technical 
means, and other factors that are related to the teacher’s personality and workplace. There are two 
references to “young in age” versus “young in spirit”. The leaders of change are usually “young in 
spirit” and also determine organizational policy. They are not necessarily “young in age”. 

Conclusion, discussion and suggestions of findings as an Entirety 

The aim of the study was to examine the teachers’ viewpoints regarding the influence of the Internet on high 
schools and the way by which they would want to include it as a teaching tool. As an entirety we can distinguish the 
following attitudes: 

1. The Internet serves as an external cultural motivator, which influences parallel internal school 
processes. 

2. A distinction should be made between distance learning via the Internet and education. 

3. The traditional school structure and the teacher’s central role in the educational process should be 
maintained. 

4. The Internet can influence teaching methods and by doing so will turn the teacher into a pedagogue 
rather than a teaching technician. 

5. The education system is responsible for the means of transferring information rather than for the supply 
of end-technologies. 

6. The teacher as a leader of change processes and as a leader of life-long learning. 

The Internet serves as an external cultural motivator, which influences parallel internal school processes 

Sharan et al (1998) describe the school as based upon a concept of community. That is to say, the 
community directly influences the learning context and processes. Indeed, a major part of the findings revealed 
community aspects and processes congruent with this concept. The influence of the Internet is reflected in social 
processes without and within the school in the following attitudes: 

• Multi-culture : The Internet enables varied communalism in the community inside the school and on the 
outside. 
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• Survival : The Internet creates new alternatives to schools, as well as new alternatives to school 
subjects. Thus, a battle is waged over the student’s heart within and without the school. 

• Censorship : There is a social dilemma concerning the ban on pornographic sites outside the school, 
and a pedagogic struggle to create barriers to such sites within the school. 

• The “information ocean” and the “giant library” : The Internet has become a relevant tool of daily life 
outside the school and a pedagogic tool within. 

• Communication : Communication, mainly asynchronic, is a significant factor in the transfer of 
information outside the school and a tool of distance teaching that can change teaching styles inside 
the school. 

The teachers perceive the very same multi-cultural environment that envelops the school as an 
unpredictable radical cultural change. A significant ingredient of cultural change is the opposition of the ultra- 
orthodox to the Internet in their schools. The telephone, which includes elements of cultural change, was not 
blocked, whereas the Internet is. Cultural change induces uncertainty that is evident throughout the study. “We are 
flooded with fragments of information about the accelerated changes we should expect without the ability to connect 
them or to establish a structured and clear pattern. . (Shalom, 1999, p. 6). It is obvious that the Internet is not a 
“product” or a process that can be isolated. 

A distinction should be made between distance learning via the Internet and education 

In the teachers’ statements, in professional literature, in the press and in the remarks of those concerned 
with education we find a confusion of terms and a dilemma, the essence of which is the difference between education 
and teaching. The discussion of findings requires us to make a distinction between these two terms: 

• Education : A social process concerned with imparting and assimilating moral values that are affined 
with society (according to the definitions of Aristotle and Dewey). “Education is a tool of carrying 
morals into effect” (Rotshtriche, 1951, p. 111). 

• Teaching : A process of imparting information and skills in a disciplinary context, in the ancient 
meaning, which states “discipline is a study subject aligned for the purpose of teaching” (Lam, 1999b). 

The present study reveals in this article that one of the significant applications of the Internet is in the area 
of teaching, as opposed to the area of education. This mainly becomes obvious when examining the interviewees’ 
statements that make a clear distinction between academic and high school studies in the first research question, 
which deals with the environmental aspects, and in the discussion of the teacher’ s role in the third research question. 
The study illustrates that a distinction should be made between the attitude to distance learning in institutions that 
are not high schools and high school teaching (Cohen, 1999). It appears that the approach in high schools is to 
student socialization, whereas universities put the emphasis on knowledge, scholarship and research. Apparently, 
teachers see themselves first and foremost as educators and the role of the high school to educate. 

Methodical inspection of virtual schools all over the world (The Virtual High School Site, 2000) shows that 
the phenomenon of distance learning is spreading - enabling the high school student not to be physically present at 
school (Salant, 2000a), but the trend is less widespread than in higher education institutions. That is to say, high 
schools seem to be educational institutions worldwide. Nevertheless, it is possible that that the growing trend of 
distance learning for high school students fulfills needs that were not examined in this work. 

According to the teachers, the socialization process - an educational one - prevents the Internet from 
replacing traditional learning. 

The traditional school structure and the teacher’ s central role in the educational process should be maintained 

Most interviewees expressed a fairly solid conviction throughout the study that high schools should be 
kept in their traditional mode with some pedagogic modifications. The modifications involve turning frontal teaching 
into experiential teaching and that the teacher move from the front of the stage to a supportive role in the learning 
process. Namely, the Internet as an extremely rich source of information will cause teachers to transfer the 
responsibility for obtaining information to the students, whereas the teacher will instruct them as to search, selection 
and verification methods of the relevant information. This is true whether teachers find social (that is to say, 
educational), pedagogic or personal justifications or whether they see the need to adjust to progress. 

Aviram (1996) discusses three attitudes to the advent of technology - the conservative, the moderate and 
the radical. It seems that the teachers in this study represent the moderate approach, which is supportive mainly of 
changes in teaching technologies and didactics. But, there also appear to be radical elements in the very wish to 
transfer responsibility to the student. It is possible that the key statement of this study is the one that states the 
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teachers’ interest in introducing the Internet into the school and refusal to take the school out to the Internet, namely 
that the schools not become virtual schools. The saying that “many things will pass from this world, but not schools: 
they will change but not cease to exist’’ (Topler, in Cohen, 1997, p. 97) is furthermore justified by the teachers’ 
attitudes in this study. 

The interviewees claim that the teacher is the center of the socialization process of youths at school and 
that “all the ills of society are on their conscience’’ (Granot, 2000, p. 18). That is, maintaining the traditional school 
structure is a reflection of the wishes of society. In both the in-depth interviews and the random ones we found a 
consensus on this issue. Social needs, youthful exuberance that must be controlled, responsibility, maturity, 
motivation and pornography (Itzhaki, 1998) serve as the central arguments for keeping pupils in school under 
teachers’ supervision. The teachers present one aspect of the profession as described by Aloni (1997). 

It is likely that the legitimate decline of school as an institution that the students must come to every 
morning derives on one hand from the cyber success of distance learning in the business world (as is apparent in the 
literature review and the press review in the appendices) and on the other hand from the fact that at present the 
student is surrounded by ideological systems additional to school that are unable to justify “education on behalf 
of. . .’’ or education “for better life’’ (Aviram, 1996). In our world, a respectable income is connected with abilities that 
involve learning and education skills (Hecht, 2000; Izenberg, 2000). This is in contrast to a survivalist role that 
dictates behaviors passed on from father to son, such as agriculture and other trades. The child’s place in school 
originates in the needs of a powerful social order (Lam, 1999b). 

It is therefore possible that a multi-directional social compromise will be reached. On one hand, the Internet 
will enable communality of students in places where they cannot come to school (Salant, 2000b). Within the schools, 
the Internet will enable communality in which the teacher coaches pupils in smaller frameworks (as described in the 
literature review), of the kind that were common the far past (Heppell, 1999). That is: “A pedagogue in Greek is a 
leader - that’s just it. Teachers should go back to being what they originally were. . .’’ (Nur - interviewee). 

The Internet can influence teaching methods and by doing so will turn the teacher into a pedagogue rather than a 
teaching technician. 

At the background of the interviewees’ positions regarding technical means, pedagogy and socialization we 
find society’s expectation of success in the matriculation exams. This in addition to the education system’s demand 
to maintain severe procedures (Sharan et al, 1998; Ben-Eliyahu, 1998). The stability and rigidity of the system and the 
failed attempts to change the structure of the external examinations can be observed in the General Manager 
Operatives (Ministry of Education, 2000), which testify to a system of constraints and balances that derive from 
social gaps and economic interests beyond the scope of this discussion. 

The interests of external interested parties put teachers to a social and personal test together with their 
students (Suan, 1997; Ezra, 1991). Matriculation exams dictate the full attitude to learning. As was found in the 
discussion of findings, teachers have no choice but to enforce a regime that enables students to cope with the 
parameters that were presented in the external perspective, which involves pedagogic aspects as characteristics of 
the external exams (the official matriculation exams). The external exams dictate major parts of the teachers’ attitudes 
to the learning materials and to their personal problems. As opposed to the interviewees’ dissatisfaction with the 
exams, Tamir (1997) states that in teachers’ view an exam produces motivation, restraint and healthy concern for the 
student, that is to say, dissatisfaction despite necessity. In their aim to guide students to succeed in matriculation 
exams, teachers harness the Internet to the task. The “giant library” described in the inner (in campus) perspective is 
the desire to bring the textbook model to the Internet. This model involves clear, undiluted, reliable and filtered 
learning materials with unchanged linear texts. This model is indicative of external exams. The textbook is a 
convenient conduit of the learning process that serves as a filter, with the world at one end and the pupil at the other 
(Horowits, 1999). 

The dissatisfaction with the external exams also creates anxiety on part of the teachers when dealing with 
the computer in the research question that addresses the personal aspects of the teacher’s status in the classroom. In 
addition to the personal psychological block, we find unwillingness to perform pedagogic processes, which require 
some form of gamble that could damage results. The fact that the interviewees perceive the informal communications 
systems via the Internet as valid only when the regular school system encounters difficulties, explains the wish to 
maintain the traditional education system that is structured to guide the pupils towards external exams. 

It is not surprising that the teachers seem to prefer a combination of papers with external exams or instead of 
them as a possible escape route, which maintains the system’s stability, enables teacher autonomy and rescues 
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pedagogy from its deterrent stability. It appears that the difference between high school learning and real-life learning 
is motivation, that is to say “learning through doing’’ (Heppell, 1999, p. 16). 

The decisive factor of learning is neither intelligence nor the didactic system, although both play some 
part in it, but motivation (Lam, 1999a, p. 170). 

As students also find greater interest in writing papers (Zilbershtein and Geva, 1992), the teachers’ opinions 
become more legitimate. “Distributed Learning’’ (Saltzbert and Polvson, 1995) combined with the statement that “the 
school will become integrated in the information sources outside if’ (Flided, 1999, p. 4) is the suggestion that is 
reflected in the interviewees’ inclination to use external Internet sources inside school. It is our impression that the 
organizational, cultural, economic and social alignment in Israel, as presented in this study, aims at this solution. 

The education system is responsible for the means of transferring information rather than for the supply of end- 
technologies 

A connection is found between the attitude towards government intervention in the external aspect and 
adjustment problems to computerized systems in the teacher’s intrinsic aspect. Our era is renowned by the fact that 
schools should supply the technology in order to improve the students’ achievements and autonomy (Peled et al, 
1989). 

This phenomenon is also evident in the US: 

“Our ... challenge is to provide Americans with the educational opportunities we'll all need for [the 
21st] century. In our schools, every classroom in America must be connected to the information 
superhighway with computers and good software and well-trained teachers. We are working with the 
telecommunications industry, educators, and parents to connect ... every classroom and every library 
in the entire United States by the year 2000. 1 ask Congress to support this educational technology 
initiative so that we can make sure this national partnership succeeds. ” ( President Clinton, US 
Department of Education, 1996, www) 

The education system is dealing with changes imposed by the supply of expensive technology (US 
Department of Education, 1996; West, 1996), the addition of standards for computer maintenance, training in 
technical topics that change with the upgrade of computers, investments in designated infrastructure for a certain 
type of computers, and the creation of social gaps that evolve due to increasing demands from the students. 

But the discussion of findings that deal with the pedagogic and intrinsic aspects, together with the evidence 
in that deals with the press review (that was part of this research presented in this article), raise the possibility that 
the characteristics of Internet technology can be disconnected from traditional table-top computer systems, namely 
systems such as cellular phones, palm pilots with Internet capabilities and other means that will be developed in the 
future. That is to say, advancing technology will enable teachers to focus on how they essentially see themselves - 
as pedagogues. 

It is possible that the remedy for this situation is privatization, which would facilitate the supply of Internet 
tools to the pupil (not the school) by financial means. At the present, the Ministry of Education or the schools 
supply the computers and they are situated only in educational institutions. Explicitly, the pupil will purchase his 
own Internet tools (just as he does textbooks, copybooks, pens and pencils), whereas the school will supply the 
teachers, the social environment and the curriculum, which will integrate Internet information sources. In other words, 
the school will nurture pedagogy - which, as is indubitably evident in discussions of all perspectives, is where the 
teachers want to find themselves. The concern expressed by the interviewees about social gaps is most likely the 
point at which the government should intervene. The fact that the pupil should supply the technology is not all 
encompassing. The supply of technical means to poorer populations is seemingly necessary. 

The teacher as a leader of change processes and as a leader of life-long learning 

We come across statements such as “training future generations for the future’’ (Shalom, 1999, p. 8) or as 
proposed by Minister Shimon Peres “. . .to teach the history of the future. . .’’ (Lecture at a conference of the Science 
and Technology Administration, Jemsalem, 2000). But this study illuminates swift cultural and pedagogic changes, 
some of which are unforeseeable. Pessig (1999), a futurist, presents us with the inability to cope with the future and 
the irrelevance of anything we might see fit to include in the curriculum. 

There is a contradiction here. On one hand, the meaning of these statements is that young people carry the 
burden of change. And, on the other hand, we find teachers to be “young in spirit’’ leaders of change. 
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It is possible to settle this contradiction if we change our outlook of education and adopt the approach that 
advocates “whatever the future may be”, we teach what is updated today, and mainly “teach our students how to 
learn” as stated Baal-Schem. At the same time, we adopt the educational approach that supports life-long learning, 
namely - instead of learning for four years, we learn for forty years (Moe et al, 1999). For, every age and every 
generation can produce leaders of change. An ability to adjust to change and strategies for the development of 
appropriate learning skills will evolve throughout one’s life. 

“Our ability to be information literate depends on our willingness to be lifelong learners as we are 
challenged to master new technologies that will forever alter the landscape of information” (Plotnic, 

1999, WWW) 

This is a realizable proposal with a learning and life program. This is a declaration that can place educators in 
a convenient and worthy position with clear boundaries of responsibility that enable examination and feedback. The 
findings reveal pedagogic ideas, which could find a solid basis in the proposed social, learning and teaching 
structure. The more creative educators could then present the fruits of their teaching and win the recognition of their 
colleagues and of the external interested parties. 

Last words 

At the conclusion of this article, it should be stated that it dealt with the teachers’ perspectives . It is their 
view that they are the ones who carry the burden and that they are the centrals factors in applying social and 
educational ideologies in high schools. The teachers want a status of professional pedagogues. The Internet is a tool 
that the teachers want to use, despite the technical and emotional constraints, because it includes pedagogic 
characteristics as well as technical ones. It seems that the Internet will generate a change in teaching methods, but 
the interviewees are certain that high schools in their present administrative format will continue to exist for a long 
time. 
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Multimedia CD-ROM Based Course Creation: Instructional Design Considerations 

for Teacher Educators 

Timothy E. Morse and Taralynn Hartsell 
The University of Southern Mississippi 
Abstract 

This paper is a report of the instructional systems design issues the lead author has confronted as he has created a 
multimedia, CD-ROM -based preservice teacher education course. The course, which introduces students to the 
field of special education, is comprised of 10 multimedia CD-ROMs that were developed solely by the lead author. 
While existing technologies enable a single teacher educator to create such a course, teacher educators need to be 
aware of the numerous issues they will have to address before embarking on such an endeavor. Consequently, the 
purpose of this paper is to identify some of the issues teacher educators will have to address if they want to 
independently create a multimedia CD-ROM-based course, and provide these instructors with some guidance for 
addressing these issues. Additionally, some of the advantages and disadvantages associated with this work are 
identified. 

Introduction 

Innovative uses of computer technology to deliver college courses have proliferated in recent years as 
access to this technology in both schools and homes has increased. In particular, online course offerings have 
become more prevalent as both the Internet and World Wide Web have become more popular. However, online 
instruction is only one type of technology-mediated instruction that can be used to present a college course. In fact, 
since no one technology will address the needs of every University instructor (Vattorn, 1998), it is necessary for 
these individuals to explore the use of any type of technology-mediated instruction that is this suited for their 
specific purposes, and then report their work so that others may learn from it. 

Existing and emerging technologies now permit college instructors to develop one type of technology- 
mediated instruction - multimedia CD-ROMs - that heretofore was not possible given the resources (both equipment 
and personnel) that were needed to create this type of technology-mediated instruction (Ludlow, 2001; Ludlow, 
Foshay, & Duff, 1998). For the purposes of this discussion, the term multimedia refers to a computer-based 
environment that utilizes multiple media (e.g., motion video, sound, graphics, and text) (Poole, 1997). For more than a 
decade special education teacher educators have used computer-based multimedia instructional materials to present 
instruction about select special education topics (Blackhurst & Morse, 1996; Langone, Malone, & Clinton, 1996; 
Semrau & Fitzgerald, 1995; Thorkildsen & Lowry, 1997). Consequently, it is appropriate to explore the possibilities of 
using this medium to present instmction for an entire college course, particularly since this medium can offer the 
convenience and asynchronous access that students often report as being an advantage of online course offerings. 

During the past two years the lead author has been independently developing a directed study, multimedia 
CD-ROM -based course: Psychology and Education of the Exceptional Individual. The course, which introduces 
students to the field of special education, is comprised of 10 multimedia CD-ROMs. As was noted above, when such 
development efforts are undertaken, they need to be reported so that others can learn from the experience. More 
specifically, since existing and emerging technologies permit a single instructor to create such a course, potential 
course creators need information that will assist them in determining whether the reasons for doing so are 
outnumbered by the arguments that can be offered against an individual completing this work independently. 

Given these circumstances, the purpose of this paper is to identify for teacher educators some of the issues 
they will have to address if they want to create a multimedia CD-ROM based course, and to provide these instructors 
with some guidance for addressing these issues. This is accomplished by first describing the multimedia, CD-ROM- 
based preservice teacher education course the lead author has created (and continues to develop since this course is 
being offered for the first time during the Fall 2002 semester). This course description is followed by a discussion of 
some of the instructional design issues the lead author has identified a teacher educator will have to address if this 
individual wants to independently create a multimedia CD-ROM-based course. The paper concludes with a 
discussion of the advantages and disadvantages associated with such an endeavor. 
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Overview of a Multimedia CD-ROM Based Course 



Rationale for course creation 

The multimedia, CD-ROM-based preservice teacher education course that the lead author created serves as 
the University's introductory special education course and is the only required special education course for all 
preservice regular education teachers at The University of Southern Mississippi. The purpose of this course is to 
teach basic information about students with disabilities and the operation of special education programs in the public 
schools. While a number of sections of this course are offered each semester at the University, the multimedia CD- 
ROM-based version was created to meet the needs of a particular group of preservice teachers who complete their 
coursework at one of the two campuses that comprise the University, which is a dual campus institution. The largest 
campus is located in Hattiesburg, while the second campus (where this course is offered) is headquartered 
approximately 75 miles south on the Mississippi Gulf Coast in Long Beach. Additionally, courses are offered at three 
satellite locations along the Coast. The preservice teacher educators for whom this course was created complete the 
majority of their classes at two locations along the Coast: either Long Beach or Gautier, which is approximately 40 
miles east of Long Beach. Since the CD-ROMs present confidential information about special education students, the 
CD-ROMs are housed in a library at Long Beach and a computer lab at Gautier in accordance with an agreement 
between the University and the public schools. 

Most of the students who attend school on the Coast are non-traditional college students. Consequently, 
the Gulf Coast faculty make a concerted effort to ensure that their course offerings meet the unique needs of these 
students. For the past three years an online version of the introductory special education course was available to all 
of the University's students. Still, some of the Coast's students requested the creation of a section of this course that 
offered them even more flexibility. The reasons for their request included not being available to attend the mandatory 
online sessions (e.g., weekly chat sessions), previous negative experiences with online courses, scheduling conflicts 
between the online version of this course and other classes the students had to take, and not having access to their 
own personal computer. 

Obtaining and creating course content 

The course is contained on a collection of 10 CD-ROMs. The CD-ROMs include video, audio, graphic, and 
text files that present: (a) the instructor's lectures; (b) special education activities being performed in local public 
schools; (c) key special education documents, such as a student's individualized education program; and (d) 
interviews with school personnel. Hence, the creation of the course was a collaborative effort between the University 
and four public school districts. 

The lead author obtained all of the video and audio source material by videotaping and creating audio 
recordings of his lectures, and videotaping special education activities being performed in local public schools as 
well as interviews with school personnel. A Sony DCR-TRV 17 Mini-Digital Video Camera and a Sony ICD-BP 100 IC 
Recorder were used for these purposes. The source material was converted to video and audio files on a Sony PCV- 
RX463DS VAIO Digital Studio computer using Sony's DV Gate Motion software to capture video and MGI 
Videowave III SE to edit it. Audio files were both captured and created using Sony's Digital Voice Editor software. 
Graphic files of key special education documents were created using an HP ScanJet 5300C and its accompanying 
software. Text files in which additional lecture materials, a study guide, and information directing the students how to 
use the CD-ROMs were created in Microsoft Word 2000. The video, audio, graphic, and text files were appropriately 
assembled as interactive multimedia instructional programs using the multimedia authoring program Lectora. These 
instructional programs were then "published" as the 10 CD-ROMs that comprise the course. 

The lead author used grant monies he secured from the University's Preparing Tomorrow's Teachers to Use 
Technology (PT3) grant to obtain all of the needed equipment and software. Likewise, the course was constructed 
during the Summer 2002 term after the lead author secured a grant from the University that expressly permitted him to 
create the course. The University annually awards "Summer Grants for the Improvement of Instruction." These 
grants release a professor from all teaching responsibilities during the summer semester so the professor can create 
an innovative way to present instruction. 

Course implementation 

Students complete the course by viewing the CD-ROMs. The students can view the CD-ROMs any time the 
facilities where the materials are housed are open. Each student must present proper identification (i.e., their driver's 
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license) to use the materials. Each collection of 10 CD-ROMs is contained in one 3-ring binder equipped with plastic 
storage pages. A total of six 3-ring binders are available for the students' use. 

The entire class met with the instructor the first Friday of the semester, and will meet with him three 
additional Fridays. During the first class meeting the syllabus was discussed and use of the CD-ROMs was 
demonstrated. Three more class meetings will be held so the students can complete examinations that pertain to the 
CD-ROMs. Additionally, each student must complete a take-home ejam that is based on the course's required text, 
and write a philosophy of special education paper. These assignments must be submitted to the instructor by their 
assigned due dates. 

The instructor and students remain in touch via telephone, postal mail, student-scheduled meetings, and e- 
mail. The instructor called each student individually during the first two weeks of the semester to conduct a follow-up 
conversation to the first class meeting. Approximately three weeks prior to each CD-ROM exam the instructor will 
send a letter to the students reminding them about the upcoming exam as well as the due dates for the take-home 
exam and philosophy paper. Students can call or e-mail the instructor when necessary, or visit him in his office. 

Multimedia CD-ROM Based Course Creation Issues 

Existing technologies now make it possible for one teacher educator to create a multimedia, CD-ROM-based 
course tailored to meet a specific need. Before engaging in such a worthwhile endeavor, a teacher educator must 
determine whether the reasons for doing so are outnumbered by the issues that must be addressed in order to create 
such a course. Potential course creators will be better equipped to decide whether, and how, to proceed with such 
work if they have a general understanding of these issues. Thus, the next section of this paper focuses on five 
central issues that the lead author has determined must be addressed when a multimedia, CD-ROM -based course is 
created. 

Rationale and stability 

The lead author learned, as have others (Ludlow et ah, 1998), that creating multimedia instructional materials, 
especially an entire course that consists of a collection of multimedia CD-ROMs, can be a very time and labor- 
intensive process. Therefore, before a teacher educator commits to such an endeavor she must first establish a clear 
rationale/need for developing the course. Simultaneously, she must determine how long this need will exist, and 
assess whether other factors indicate that, once created, the multimedia course will be used enough to justify the 
effort associated with its creation. Relevant questions that need to be asked at the outset include: (a) will enough 
students enroll in the course each time it is offered so that the class will not be canceled; (b) will the course's creator 
be permitted to teach the course each time it is offered, and not be bumped by a more senior faculty member or re- 
assigned to teach another course that requires the course creator's expertise; and, (c) will the major issues and 
content that are taught in the course remain relatively stable and only have to be revised/updated periodically? A 
related matter that must be addressed at this time is whether necessary course updates can be completed with the 
resources and personnel who will be available at that time. 

Equipment 

An array of equipment (e.g., cameras, videotapes, microphone, computer) and computer software will be 
needed to create the course. Afterwards, appropriately equipped computer systems will be needed for viewing the 
course. One key issue that course creators should attend to from the start is the seamless operation of all of their 
assets (i.e., will every piece of equipment and software be compatible with every other piece of equipment and 
software). Manufacturers may advertise that their equipment possesses certain multimedia capabilities, but these 
capabilities may only work when the equipment is paired with other equipment that is made by the same 
manufacturer. Thus, mixing and matching equipment from different manufacturers may prove to be problematic. 
Another advantage of achieving seamless integration by purchasing all equipment from the same manufacturer is 
that the course creator will only have to go to one entity to pose questions, as well as to report problems. A course 
creator may have to pay more for a particular item by sticking with one manufacturer, however, for all of the reasons 
just discussed, this may be a cost worth incurring. 

A second equipment consideration pertains to how the multimedia CD-ROMs will be viewed. Relatively old 
computers may not possess the processing speed, RAM, and screen resolution that is needed to view full motion 
videos as they are intended (and expected) to be seen. Hence, students who use these computers to view the 
course's CD-ROMs may be thwarted in their attempts to achieve the learning objectives and become dissatisfied with 
the class. To address this issue a course creator may have to designate a central location, such as a computer lab. 



388 




where students can go to view the CD-ROMs. If a course contains confidential material that is restricted for the use 
of only the students who are enrolled in a course, this fact may dictate the need to locate a course in a central 
location. 

Funding 

Funding may be needed for several purposes. First, funding may be needed to obtain the necessary 
equipment (e.g., hardware and software). Course creators should be mindful of the fact that, even though they may 
prefer to purchase top-of-the-line equipment across the board, nowadays they will be able to complete their work 
with much less expensive equipment. The key is to purchase equipment that meets one's needs and then get the most 
out of it by becoming an expert user of it. 

Second, funding may be needed to purchase release time for the purpose of obtaining source materials . This 
should include the time one will have to spend planing exactly what source material will be obtained and how it will 
be obtained. For instance, before conducting an interview one should settle upon the questions that will be asked, 
the order in which they will be asked, and the setting in which the interview will take place. Related expenses that will 
be incurred include the cost of transportation for going to and from the University and the school (or similar location) 
where the source material will be obtained. 

Third, funding may be needed to purchase the release time that will be necessary to construct a course. This 
involves placing the source material in appropriate video, audio, graphic, and text files, compiling these files in the 
multimedia authoring program, burning the CD-ROMs, and constructing the 3-ring binders (or similar storage device) 
that will hold the CD-ROMs. 

Source material acquisition 

When a potential course creator considers the material she wants to place in the CD-ROMs that will 
comprise her course she will be well served to remain mindful of the statement, "Content is king.” While multimedia 
offers the possibility of generating lots of "bells and whistles,” these entertaining features may only serve to distract 
the learner fromattaining the desired learning objectives. Thus, the main focus of the course creator should be to 
present quality information that is tied to the attainment of stated learning objectives . 

In their quest to obtain the highest quality content possible teacher educators must plan for how they will 
obtain source materials ethically. Since teacher educators need to teach preservice teachers about what is occurring 
in schools, the employees and students who populate the schools will be the centerpiece of the source materials and 
they, or their legal guardians, will have to grant permission for their participation in the project. When a course 
creator obtains this permission, she will be best served if she explicitly states how the source material is to be used. 

In the lead author's project, he drafted participant permission letters and had them approved for his use by the 
University's legal counsel. Potential course creators are encouraged to do the same. 

Undoubtedly, addressing all of these issues will entail a lot of work. In the end, however, this work will, more 
likely than not, result in a quality product. Additionally, strong, long-term working relationships will be established 
with the school personnel who contribute to the project. 

Creating computer files 

As was noted previously, software will be needed to create the video, audio, graphic, and text content files, 
as well as compile these files in the interactive multimedia instmctional programs. Given the central importance of this 
software to the project, it should be relatively easy to learn how to use (in other words, the operation of its features 
should be intuitive), stable (free from frequent crashes), compatible with the hardware and other software the course 
creator is using, and be able to work across platforms (i.e., Windows and Macintosh). This last point is particularly 
relevant to the multimedia authoring software program. 

Advantages and Disadvantages to Self-Made Multimedia Instmctional Materials 

Ludlow et al. (1998) have argued in favor of using a development team to create multimedia instructional 
materials, saying that the use of such a team allows it to create better materials since each team member can use his 
expertise to enhance the development of these materials. Yet, existing technologies now permit individuals to 
independently create multimedia instructional materials, and there are advantages associated with independent 
production as compared to the use of a development team. Below is a brief discussion of the potential advantages 
and disadvantages of having one person -in particular, the course's instructor - independently create a multimedia 
CD-ROM based preservice teacher education course. 
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Advantages 

Scheduling course creation activities. As has been noted elsewhere in this paper, a number of activities 
must be performed to create a multimedia, CD-ROM based preservice teacher education course, including 
videotaping school activities, creating needed video, audio, graphic, and text content files, and creating and revising 
interactive multimedia instructional programs. These activities may be much easier to schedule and complete when 
the course creator works independently rather than as a member of a development team. 

Obtaining course content. If the course creator is also the content area expert, when she obtains all of the 
source material independently - particularly video and audio - there is a greater chance that she will obtain content 
that is particularly suited to her needs. For instance, since this individual knows what she wants to highlight in a 
video, she can be sure to capture this information in her source material. Later, when she creates the video, audio, 
graphic, and text content files, she can take necessary steps to emphasize key points displayed in the source material. 

Course creation. When the course's instructor is developing the interactive multimedia instructional program 
she will be in a position to determine how best to present and integrate/link the content. These actions may occur as 
a result of systematic planning or perhaps by happenstance as the course creator learns about the multimedia 
authoring software's capabilities during the course creation process. 

Professional development. By going to schools to videotape lessons and related activities, and interview 
school personnel, the course creator will be participating in a professional development activity throughout the 
course creation process. 

Build University /public school partnerships. When a University instructor works with public school 
personnel to create multimedia instructional materials, she will have the opportunity to establish a long-term, positive 
working relationship between the two entities. 

Disadvantages 

Time and resource commitments. Multimedia CD-ROM -based course development is a time and labor- 
intensive process. Thus, at the outset the course creator must determine whether she will have the time necessary to 
devote to all aspects of the project. University personnel in particular must determine whether their work in this 
regard will be valued by their employer and counted appropriately towards tenure and promotion. 

Technical skill requirements. A single course developer will need to possess quite a number of technical 
skills across many areas, including operating a video camera and audio equipment, as well as computer hardware and 
software. Noteworthy deficiencies in this regard may result in (a) the acquisition of poor quality source material, (b) 
the production of poor quality video, audio, graphic, and text content files, and (c) an inadequately developed 
multimedia instructional program (e.g., inadequate links between related concepts and limited interactivity). 

Instructional design training. If a course developer has limited or no training as an instructional designer the 
resulting course may present less effective instruction than would otherwise be the case. Areas in which an 
instructional design background would prove helpful include (a) how to best use video and audio to enhance 
student learning and (b) multimedia CD-ROM design, to include individual screen designs, sequencing of content, 
and navigating among the content. 

Conclusion 

Existing and emerging technologies now permit teacher educators to create multimedia instructional 
materials tailored to meet their particular needs. For instance, teacher educators can create a multimedia CD-ROM- 
based course similar to the one that was created by the lead author and reported upon in this paper. However, given 
the numerous issues that need to be addressed in this type of project, the fact that existing technologies permit this 
type of development does not mean that the reasons for doing so will outnumber the arguments against doing so. 
Hence, potential course creators are encouraged to explore all course creation issues, including the ones addressed 
in this paper, before deciding on the appropriateness of creating and using this form of technology-mediated 
instruction. 
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Directing a Dynamic Student Managed Technology Lab 

Juanita Ikuta 

University of Wisconsin Eau Claire 
Purpose 

The purpose of this manuscript is to offer practical experience from the University of Wisconsin -Eau Claire’s 
School of Education Technology Lab and Projects Studio (referred to as the Lab) and research-based information on 
the need for and the management of student run, self-contained computer labs. 

Intended Audience 

This manuscript is intended to provide practical information on justifying and managing a student run, self- 
contained technology center. This information may be used to discuss individual approaches with others and as a 
catalyst for examining alternate systems. The information is intended for higher education administration and faculty, 
along with K-12 administrators and media specialists. 

Content Description 



Justification 

The concept of a education computer lab is well-founded in research. One study identifies recognition of 
need for pre -service teachers to have a dedicated environment for learning technology skills. This study found that 
the acknowledgement of the relevance of information indicates a progression in technology adoptions (Oleson, 
2000). Another study concludes that pre-service teachers using technology in a supported lab show an increase in 
computer usage, express a higher level of confidence in using computers and indicate that they would use computers 
more extensively in the future (Castleman, 1995). Yet another study, focusing on integration, indicated the need for 
more hands-on experience for pre -service teachers coupled with increased modeling of technology. (Panel on 
Educational Technology, 1997; Office of Technology Assessment, 1995). All this research suggests that a pre- 
service teacher computer lab with trained support staff increases confidence, skills and may impact future integration 
of technology in the classroom. 

Strategic Planning 

The foundation for building a pre -service teacher computer lab is a strategic plan. It is important to devise 
mdimentary components of a strategic plan, revisit and carry them out. Lack of a collective vision may result in the 
inability to prioritize an overall operational mode during changes in administration. Varying operational styles may 
exist as coordinators, platform administrators and staff change from semester to semester. Operational styles could 
range from casual, unstructured operation to more procedural and structured. A lack of consistency could unsettle 
staff resulting in an aversion to change and the emergence of power struggles amongst staff. 

Roles 

Another important element in a student managed technology lab is the roles that provide consistency from 
semester to semester. Suggested roles include: faculty liaison, coordinator, PC administrator, Mac administrator, 
patron relations’ software specialist administrator, assistant administrators and lab workers. The reporting structure 
and degree of responsibility assumed at each level should evolve and responsibilities may emerge as a technology 
lab matures . 

Recruitment 

Einding students to fill technology positions is a key issue for any computer lab. Several strategies at the 
Lab were successful. Eirst, collaboration with a financial aid representative to determine students with federal work- 
study money identified potential candidates that do not have to be paid from lab funds. Second, professors teaching 
technology classes were a rich source of possible candidates. Professors recognized students with potential and 
interest and informed them of employment opportunities. Other recruiting was performed by hosting open houses , 
giving presentations in selected classes and dedicating a page on a web site. 
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Staffing 



Successful recruitment efforts are essential to providing adequate staffing for dedicated support. Research 
indicates that when teachers are given dedicated support they allocate time to develop technical expertise and plan to 
effectively integrate technology into their classrooms (Colburn, 2000). One quantitative staffing method Lab 
administrators used analyzed the number of patrons on an hourly morning, afternoon and evening shift and daily 
basis. Peak usage times were identified and staffing numbers adjusted. 

Training 

Just having a staff is not enough - a trained staff is a necessity. Two training methods for Lab staff have 
been tested in the Lab. Initially, a learning matrix was established. This matrix listed frequently used computer 
programs with sequential skill levels. Lab staff was responsible for mastering specified skill levels to obtain pay 
raises. This style failed because minimal pay raises did not entice staff to advance in the matrix nor was there any 
effort made to consistently encourage the staff to use the matrix. 

A second method began with gathering syllabi from professors with technology related assignments and 
assembling a table that indicated what the assignment was, when it was due and the relevant professor. This tool 
provided a quick reference so staff could anticipate patron needs. In order to ensure that the staff had the necessary 
technical skills to assist patrons, training was prioritized according to technical skills required for assignments and 
the due dates for these assignments. The emphasis was on recognizing that training systems can rapidly become 
stagnant and the Lab must be able to quickly change training priorities to meet changing needs and technologies. 

Focus Groups 

Keeping abreast of the dynamics of technology is challenging. Reports indicate that many teachers are not 
making effective use of technologies that are present in classrooms and lack the necessary expertise to integrate 
technology into curriculum instruction (Colburn, 2000). To address this issue. Lab interest groups were created to 
increase the staffs technical expertise. Accessibility, Equipment/reservation Check-Out, Learning Matrix, Web and 
Lab Modernization comprise current focus groups. 



393 




How the Webmaster Almost Lost His Job: Reflections Upon a Usability Study for 

a Health Care Consortium 

LeAnne K. Robinson 
Western Washington University 
Abbie Brown 

California State University, Fullerton 

Abstract 

To determine whether a Web site was poorly designed or the users of the site had inadequate skills to make proper 
use of the site, a test of the Consortium of Multiple Sclerosis Centers Web site's “usability” was constructed and 
conducted over a five-month period. 

A formal usability study of the “Professional Resources” section of the MSCare Web site (www.mscare.org) was 
conducted between September, 2001 and January, 2002. A usability study is an evaluation of how well a design 
works in practice. Usability testing is a standard, time -tested evaluation protocol among engineers and has been 
adopted by software developers (beginning in the middle 1980s) as a means of determining how well a design 
works in practice. This particular test attempts to determine if the “Professional Resources” portion of the Web 
site, MScare.org works as it was designed to, to be accessible to the general public, if a user’s hardware limits 
accessibility to portions of the Web site, and to begin to understand why individuals may not be participating in 
the Virtual Roundtable portion of the site. 

Follow up questions and tests of the site on various computers showed that, although the site is generally well 
regarded by the test subjects for making content of its kind available on the World Wide Web, the navigational 
setup and links, aspects of the visual design, as well as security protocols currently in place within the 
Professional Resources section need to be re-examined before the site is able to meet the goals outlined by the 
CMSC Web Site Management Team. 

Introduction 

“I must be doing something wrong.” This statement and others like it seem to be uttered regularly by people 
attempting to navigate their way through the World Wide Web. Many people seem willing to blame themselves when 
it comes to using computing tools, instead of considering the possibility that the tools are poorly designed. The 
authors of this study found themselves in casual conversation with health care professionals who, while blaming 
themselves for their inability to access information on a Web site that was important to their professional growth and 
their patients’ well-being, seemed to suspect that something else might be affecting the situation. As researchers, the 
authors were intrigued and interested in discovering more about why these highly educated and competent people 
were made uneasy by software designed especially for them. 

After continued discussion with the health care professionals (who ultimately became the clients of this 
study) the question to consider became: Could a usability study reveal whether the problems reported in using a 
specific Web site were the result of the users’ lack of computing skill or the result of poor software design? 

The Client 

The MSCare Website has been developed for the Consortium of Multiple Sclerosis Centers (CMSC) as a 
vehicle for disseminating information to its members, MS professionals, patients and the general public. The 
organization was founded in 1986 under the direction of neurologists who were interested in the clinical care of 
multiple sclerosis (MS). Since then, the organization has developed into a multi-disciplinary organization providing a 
network for all healthcare professionals and others specializing in the care of persons with MS. A management team 
was formed in 1999 to oversee the development of a Web site and to insure that the site reflects the mission and 
goals of the CMSC. 
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The MSCare Web Site 

The MSCare Web site has been divided into three sections providing information for: CMSC members, 
patients, and professionals. Professional Resources is a subsection of the Professionals’ section and has been 
created to serve primarily as a resource for consortium members and secondarily as a resource for professionals 
interested in MS patient care. The Professional Resources is divided into four areas: On-line CME’s . On-line MS 
Presentations . Book Reviews and Articles on MS by CMSC Affiliates . The Virtual Roundtable , which was initially 
included as part of the Professional Resources section, was moved to CMSC Members section during the evaluation 
period. 

On-Line CME’s include several links to sites where individuals can take classes via the Internet for 
continuing medical education credits. Articles on MS by CMSC Affiliates are a series of papers presented at 
conferences, written by Consortium Members or by other professionals. 

The On-Line Presentation section consists of several presentations that were given at recent conferences 
and are presented in an outline and notes format. A narrative outline is generally on one half of the screen and the 
notes elaborating on the points presented are on the other. Posted presentations include a multitude of formats, some 
requiring Real Player (www.real.com) installation for video viewing of the actual presentation. 

The Virtual Roundtable was created as a forum where consortium members can log on and participate in on- 
going discussions regarding the posted article. All viewers are able to read the article, but only registered members 
may participate in and have access to the on-line dialogue. The intent of the selected article is to provide thought 
stimulating discussion around professional interests. 

The director of the Web Site Management Team, who is also editor of the Professional Resources section, 
requested that the Professional Resources be tested for usability. The primary evaluator held an initial interview with 
the director of the Web Site Management Team on November 11, 2001 to gain an understanding of the evaluation 
objectives. From the conversation, several evaluation questions were posed and then confirmed by the director. 

Method 

A participant-oriented program evaluation design (Worthen, Sanders & Fitzpatrick, 1997) was modified to 
meet the client’s evaluation objectives. Although the original request was for a usability test of the Web site, 
discussion between the primary evaluator and the Web Site Management Team director revealed that not all of the 
client’s evaluation objectives could be met using standard usability testing procedures. The client had identified at 
least one objective that a usability test cannot help to answer: the question of who is using the site cannot be 
answered through standard usability testing (in discussing the issue with the site’s Webmaster, it was determined 
that no information is currently available as to who exactly has visited the Web site). Only the first objective, 
determining whether or not the site is usable and accessible may be determined using standard usability testing 
procedures. Another of the client’s objectives may be answered in part through standard usability testing protocols: 
determining why there had been no participation in the Virtual Round-Table may be achieved by determining whether 
or not the Virtual Round-Table functions as it is intended to and is understood as such by site visitors (however, if 
the testing reveals that it functions correctly and is understood, the reasons for why it is not being cannot be 
determined). 

Few pre -created formats or protocols that address client-centered requests are currently available in the 
public domain. Checklists and generic rating forms that are available do not seem appropriate as a valid means of 
accessing the desired information requested by the client (Dumas & Redish, 1999; Nielson, 2000; Small & Amone, 
2001). Thus, a formative, client-centered evaluation approach was selected and evaluation was designed around the 
client’s requests. The validity of this approach was checked by the primary evaluator in consultation with a 
university professor who specializes in usability testing. Once the plan was determined as valid, specific protocols 
were developed for use. 
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Evaluation Plan 



Table 1 : Evaluation Plan 
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A final, and crucial, question posed by the client is. Will people return to the Web site if they have difficulty 
using it? A standard presumption is that people do not return to a Web site if they find it difficult to understand or 
navigate (Nielson, 2000); nor will they return if the site design is such that the user’s computer cannot support the 
software employed by the site (e.g. javascript, shockwave files. . .etc.) (Nielson, 2000). There are exceptions to this, 
particularly if there are no other sources of reliable information available or if visiting the Web site is part of one’s 
vocational or avocational responsibility. Assuming that MS professionals have other, similar sources of information 
available and that the majority of the site’s visitors do not have any obligation to use the Web site, it can be argued 
that finding the site difficult to understand or use will cause users to avoid the site in the future. Conversely, the fact 
that the information is available on a Web site and accessible through a personal computer may have induced a 
“novelty effect” that causes users to return to the site regardless of how easy the site is to use. 

Ten test subjects were asked to participate in the usability testing. Subjects worked in pairs - this is a 
recommended protocol for usability testing since pairs need to speak aloud to each other in order to negotiate the 
tasks involved in the evaluation, allowing the evaluator to “listen in” on the subjects’ thought processes. Four of 
the pairs were selected because of their medical background and the fifth pair was selected based on their advanced 
ability to use the computer for the purpose of comparing and validating the results of the testing. Two pharmacists; 
three RN’s; two LPN’s completing an RN degree and one special education instructional assistant comprised the 
initial teams. All of the subjects had some, if not extensive, experience working with individuals with MS, with the 
exception of the final pair, who were both graduate students and “Super Users” (a common term used by Information 
Technology professionals to identify individuals with a high degree of computing sophistication; not necessarily 
programmers, these are people with extensive skill using computer applications). 
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Testing occurred in private homes between January 5 and January 20, 2002. Each pair was given a list of 
tasks and follow up questions that were to be answered after the completion of the test and were observed by the 
principal evaluator. Notes were taken on the length of time pairs spent on each task and on the perceived difficulty 
or ease of completing each task. Dialogue that occurred between the individuals was also recorded. Each participant 
received a $25 gift certificate from a national bookstore chain and an MScare.org Post-It pad as a thank-you gift for 
completing the test. 

After all of the tests were completed, the data from each test was compiled into a comprehensive set. Both 
positive and problematic comments and observations were coded and then matched with a usability checklist. 
Responses to the follow up questions were consolidated and contrasted with the results of the observations from the 
actual testing. Responses from the observations made by the principal evaluator were matched to the follow-up 
responses from the subjects as a means of triangulating the data. It should be noted, however, that most individuals 
attributed problems with the web site to their own computer skills or to the computer that they had purchased, not to 
the site. Even the Super Users, who had difficulty with navigation gave positive comments on the ease of using the 
site. Users who could not find information, or answered informative questions incorrectly concluded with such 
comments as, “Seemed to have a lot of useful information for someone who uses a computer’’. Both evaluators then 
tested the Web site for navigability issues related to color, screen size and links on both Macintosh (MAC) and PC 
platforms in Netscape Communicator 4.7 , Internet Explorer 5.0 and America Online (4.0 on PC; 5.0 on MAC). Data 
from all three sources was again consolidated. The site was also tested for ADA (Americans with Disabilities Act) 
compliance using Bobby (a standard evaluation protocol available free of charge). Although this was not specifically 
requested by the client, the information was included for consideration. 

Findings 



Interface and Navigation 

Interface and navigation refers to how the set up of the site assists the user in moving from page to page 
and how the structure contributes to the ease with which a user is able to access information. All of the users were 
able to understand what the site was designed for, and several commented on how much they liked the division of 
information in the home page: Information for Patients, Professionals, and CMSC Members. No comments were 
recorded on either a cognitive match or disconnect between the content and the personality of the site. 

Numerous users had difficulty with window size. Several pop-up windows would not enlarge during the 
testing and the users had to scroll through a very small area in an attempt to complete tasks. It was also noted that 
the pop-up windows would sometimes entirely cover the previous window, and, without navigation clues, users 
could not figure out how to return to the previous page. This was often exacerbated by the unavailability of the 
“back’’ button in the browser window. 

Eurther confusion resulted when the majority of users were unable to figure out where, in the site they were; 
when the users were asked to go to “Presentations on MS’’, they were not always able to determine if they were 
actually in the correct section. This was also compounded by the incongruity among the wording of the home page, 
the left frame and the Presentation Index. Eor example, on the initial page for Professional Resources, the links are 
named Articles on MS, Online MS Presentations, and Online CME’s. On the left frame, however, the wording 
changes to: MS Articles, Presentations, and CME Online. This is further confounded by a link that appears 
underneath several of the online presentations entitled “Presentation Index’’. Users did not always know where this 
would go and were hesitant to use it. When they did, multiple problems resulted. 
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Links 

Because users were unable to easily determine how to navigate, many would completely close out of the 
program and start again by retyping the address. Sometimes in an attempt to navigate they would inadvertently 
close the program. Three of the initial four pairs were unable to locate the “home” icon. 

The Presentation Index link in Angela Chan’s article, “Journal Publication for MS Professionals” did not 
work on any browser in any platform. It often resulted in the display of a “New Site Certificate” dialog box. Several 
links that failed repeatedly were “Evidence Based Treatments for MS”, “Swallowing Disorders”, “Emerging Role of 
Nursing”, and both articles on “Thinking Critically about Web Resources”. Often when one of these was opened 
and the user returned to the index, the page could not be re-opened. 

There was no consistent distinction between general labeling and the links. Sometimes links were 
underlined, sometimes they weren’t. Links were confused with underlined blue headings that were intended to show 
the users where they were within the content. This resulted on two separate occasions in random clicking by the 
users. The Slide Navigator in the On-Line Presentations was difficult for users to follow. They were not sure how 
to move from one slide to the next and did not always readily grasp that the slides could be changed by clicking on 
the arrows and often attempted to click on the bullets in the actual presentation. It was also noted that the numbered 
links were not all available in the presentation. 

Only one user pair was able to access the Virtual Office Roundtable article, and, even when doing so, had 
great difficulty finding the article or viewing the posting. As these were the Super Users, it is likely that any other 
users would have had the same, if not more, difficulty using this section. 

Text 

One subject commented numerous times on the difficulty of reading the small print. Re-examining the 
subjects’ demographic information revealed this individual was significantly older than the rest of the sample 
subjects. Print size may need to be adjusted to be easily read by all ages of the population and to meet ADA criteria. 
Two of the pairs commented on the periodic use of blue text on black background that often appeared in a navigation 
bar. Blue text on a black background is widely recognized by design specialists as a particularly unreadable 
combination. This was also noted when both evaluators conducted follow-up tests. 



Graphics 

Graphical use was limited. The one picture that was used, (a photograph of an article’s author Patricia Cole), 
was clear on the PC in both Internet Explorer and on Netscape Navigator. On other platforms, however, the picture 
was pixilated or had strange colors, such as green spots. 

Content 

One pair commented on the wordiness of the articles. Most of the pairs reported on how much they liked 
the presentations, even though there were issues with the slide navigator. Three of the four groups commented on 
how much they liked the organization of the On-Line Article section entitled “Web Resources”. 
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As stated earlier, users did not always recognize where they were or what they were viewing. For example, 
Patricia Cole’s “Gender Issue in MS’’ is located under both On-Line CME’s and On-Line Presentations . When users 
came across the same link a second time, they questioned themselves and often left the site. 

Multi-Platform Testing 

Both evaluators examined the links on multiple computers using the three most popular Web browsing 
tools: Internet Explorer, Netscape Navigator and America Online. Minimum configuration standards were agreed 
upon by the two evaluators that included the following technical specifications: Monitors set to 256 colors (the 
lowest common denominator setting for all color monitors); 800x600 pixel screen size (the standard size of a laptop 
computer. Results of the testing were provided to the client. The main problem common to all platforms was the 
appearance of New Site Certificates. 

New Site Certificates 

The appearance of New Site Certificates caused great concern for several users. When they appeared, users 
quickly closed to the site and created a fear for further site use. One user shouted, “New Site Certificate! This is 
baloney.’’ after having encountered them several times and then refused to continue in the Virtual Office section. 

New Site Certification is intended for use with authentication and identification of users when confidential 
information, such as credit card numbers, is being transmitted. It was unclear to the evaluators the need for such 
security. 

Recommendations 

Evaulation Question 1: Is the general public able to easily access information on the web site? 

As stated earlier, the usability test was completed by ten subjects in groups of two (five pairs). The fifth 
pair were Super Users, atypical of the general public, and used for evaluative purposes to see how problematic initial 
findings were. The following recommendations are given in list of importance in assisting the general public; most of 
whom are not Super Users. 

1. Problem: Inconsistent Labeling of information between home page, left frame 
index, sites and individual links. 

Solution: Consistent language should be used to identify all of the pages and for navigation. Eor example, 
the link Articles on MS by CMSC Affiliates on the first page of Professional Resources should be identified 
as Articles on MS in the left frame as opposed to MS Articles . 

2. Problem The home icon is consistently not recognized 

Solution: Change the icon. It was not recognized as a “home’’ and created many navigational problems for 
all users. The use of a symbol alone is also inconsistent with the rest of the navigation, which is in text 
format (easiest solution would be to generate a text link that says “MScare Home Page’’). 

3. Problem Numerous Dead Links 

Solution: Correct or remove them. Dead links were deemed very frustrating by the subjects and are 
generally considered problematic by design specialists. In this case, the subjects became confused and 
would close out of sites and site extensions. Correcting or removing dead links may help to eliminate many 
of the navigation problems. 

4. Problem Inconsistent ability for user to return back to previous page. 

Solution: Create a persistent navigation bar that allows the user to return to the previous page, or use 
consistent wording and available links so that users can return easily to the previous page or the menu that 
they started in. 

5. Problem Readability of some links and identification of links 

Solution: Change black background with blue text links that appear in some pages. Many users do not see 
that these links. Linking text color should differ from non-linking text. Currently, all headings are blue and all 
links are blue. Some headings are underlined, some are not. Some links are underlined, some are not. 
Application of a consistent standard that identifies and differentiates links, viewed links and content 
headings should solve this problem. 

6. Problem Pop-up windows cover previous page entirely or are too small to easily view the information and 
cannot be adjusted. Users become lost and confused. 
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Solution: Re-size new windows so that users may easily return to the previous section and make sure that 
windows are large enough to scroll through a sizeable amount of information. 

Evaluation Question 2: Does a user’s hardware (computer, browser, etc.) limit accessibility to the posted information? 

After testing the site in six different configurations, it appears that accessibility limitations are due more to 
site design, as opposed to a users’ hardware or software. Most of the issues observed occurred cross-platform and 
may be solved with addressing the problems outlined in the previous section. There are some minor issues with 
graphics that were ide ntified on a few MAC platforms. (Charts with spec ifics were provided to the client). 
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Figure 2: New Site Certificate 

One problem that occurred at different places on different platforms was the appearance of Site Certificates. 
General users quickly exited out of the site when these appeared. In some situations, the evaluators were able to 
accept the certificates and move forward, in others, the evaluators were not allowed to view the contents of the site. 
This security protocol should be removed if the intent is for the site to be available for public use and is not being 
used to send and receive confidential information. 

Evaluation Question 3: Why aren’t professionals participating in the Virtual Roundtable? 

The initial evaluation plan included time to interview consortium members. It was later determined that 
interviewing members was beyond the scope of this evaluation. However, results of the Virtual Roundtable section’s 
usability can be addressed. For the majority of the evaluation, users were not able to access the Virtual Roundtable. 
The one pair that could was unable to open the article. When the Super User’s came to the Virtual Roundtable task, 
they were able to open the article, but had a very difficult time determining how to read the postings. While the 
client’s concern was initially over the topic, the ease of use of the Virtual Roundtable should be addressed and a 
separate and thorough test of usability should be conducted prior to questioning the topic. The evaluators postulate 
that a Roundtable that is not easily used, or whose purpose is not clearly identified, will not be used no matter how 
interesting the topic. 

At the conclusion of this report, the link the Virtual Roundtable (Virtual Office) led to a separate site, 
(www.mscare.net) and was no longer available. 

Discussion 

The MsCare Web Site has some highly appealing characteristics and has the potential to be an incredibly 
informative and oft -visited site. Participants all commented on wanting to return to find information and several 
commented that they had wanted resources that they might refer to family and patients. The Profession Resources 
section, which is the focus of this evaluation, was often difficult to navigate and users became frustrated and bogged 
down. Their interest in viewing the site diminished as they could not complete simple information finding tasks, and 
most users began to blame themselves, a lack of computer skill (which was not generally the case, most had very 
good general Internet skills), or their computer. If a site is difficult to navigate it is likely that users will not return or 
use the site consistently. 

Some of the Professional Resources components that the users expressed interest in were difficult to find. 

In particular, the initial four pairs all were delighted to find “Web Resources for Professionals’’, but, when the testing 
was over, most were not able to relocate this spot. With some slight reorganization, and a minor re-design of some 
navigational elements, the goals outlined for the MsCare Professional Resources Web Site should be easily met. 

Conclusions 

The original hypotheses was confirmed: a formal usability study indicated whether a specific Web site (in 
this case, www.mscare.org) was inherently difficult to use, or whether some of its regular users lacked the skills 
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necessary to make use of the site. The study revealed flaws in the site’s design, which served to explain why the 
users had felt themselves to be personally inadequate to the tasks involved in accessing information on the site. 

This study further demonstrates the need for continued development and implementation of usability 
testing protocols in software design. Were it not for the study, the constituency of the MSCare organization might 
well continue to blame themselves for their inability to make use of the World Wide Web, causing them to feel 
disenfranchised from this powerful medium and making them feel inadequate to the tasks of navigating through and 
retrieving information of importance to them as professionals, and to the patients who rely on their knowledge. 

Furthermore, results of this study indicate that a usability study team may be of tremendous help to 
Webmasters and software developers by allowing them a means to tailor their designs to the needs and skills of a 
particular user group (in this case, medical professionals with moderate computing skills). The results of this study 
strongly suggest that usability testing is critically important part of creating useful software, and that more research 
on the protocols and practices of usability testing are both required and recommended. 
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Addressing NCATE Standards for Technology, Pedagogy, and the Curriculum 

Heidi L. Schnackenberg 

Margaret D. Maughan 

This paper outlines one university’s attempt to integrate technology and pedagogical practices. Through 
the interweaving and co-teaching of curriculum and media courses, the college of education attempts to meet the 
National Council for the Accreditation of Teacher Education (NCATE) standards. The combination of these courses 
demonstrates how various computer and peripheral electronic tools can be utilized to support teaching practices, 
subject matter instmction, and overall classroom environment. The courses were taught both in a computer lab and a 
non-computerized classroom. Eaculty and candidates interacted with the technology, and shared lesson ideas, 
classroom management techniques, and pedagogical practices. Twelve Adolescence/Health Education majors were 
enrolled in the combined courses. 



Overview 

Plattsburgh is one of 13 university colleges operated by the State of New York. The 64-member system also 
includes community colleges, technical colleges, and doctoral degree granting institutions. The campus is located in 
a historic, rural area in the northeast corner of New York, situated on the western shore of Lake Champlain, east of the 
Adirondacks and about twenty miles south of the Canadian border. The greater Plattsburgh area contains about 
40,000 people. Pounded more than one hundred and ten years ago as a “Normal School” or teacher preparation 
college, SUNY Plattsburgh has expanded to offer undergraduate degrees in nearly fifty academic disciplines with an 
undergraduate enrollment of 5,048 students. There is also a sizeable graduate program in Education. Within the 
School of Professional Studies, there are four specialized courses of study: Elementary Education, 
Adolescence/Health Education, Special Education, and Literacy Education. The number of students enrolled in these 
programs total 1,328 or about one in six students at SUNY Plattsburgh. The faculty is substantial, including 29 full 
time instructors and 44 part-time instructors (Plattsburgh State University, 2002). 

The Adolescence/Health Education program at Plattsburgh State University requires a core course entitled 
“Curriculum and Instruction for Secondary Schools.” It provides intensive training in the planning and 
implementation of instruction at the middle and secondary levels. Utilizing the state and national standards, students 
formulate behavioral objectives to address a wide range of cognitive and affective processes. They construct 
instruments to measure intended outcomes of instruction and design teaching plans that they implement in a 
classroom setting during their field experience. Instruction includes the areas of critical and creative thinking as well 
as problem solving. Students learn to adapt materials to meet individual differences and levels of ability. 

Another required core course for Adolescence/Health Education majors is EDU 359, “Integrating 
Technology into Teaching Practices.” Candidates in this class earn one-credit, which is 15 hours of contact time. 

The class is based on the incorporation of technological resources and tools into pedagogical skills. In the course, 
candidates develop and implement lesson plans that utilize technology to support instruction and learning. The 
implemented lessons are then videotaped so that candidates can watch them at a later time and self-evaluate their 
teaching skills. The instructor meets with the candidates regarding the self-assessments and together they identify 
areas of success and strategies for improvement. Web-based showcase portfolios are als o developed by the 
candidates in order to illustrate their learning and accomplishment in the overall degree program. Einally in this 
course, with the cooperation of Plattsburgh State University Student Support Services, adaptive and/or assistive 
technologies are discussed and explored. 

The ways in which EDU 395, “Curriculum and Instruction for Secondary Schools,” and EDU 359, 

“Integrating Technology into Teaching Practices” meld technology, teaching, and learning are numerous. Prom the 
teaching perspective, the course instructors frequently use Power Point to illustrate lectures and the Internet to 
supplement information given in class. Also, instructors communicate with candidates via email to notify them of 
resources, events, or homework. (These notices are sent as attachments as well as in the body of the text.) As well, 
instructors capture good and bad examples of candidate “teaching moments” (with candidate permission) using a 
digital camera. These pictures are then used as the basis for discussion about effective or ineffective teaching 
styles/techniques . 

Candidates utilize technology as a part of their coursework and professional development as well. World 
Wide Web tutorials and assignments are used as in-class group and/or individual activities to reinforce concepts. 
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skills and ideas covered in the two classes. Candidates are required to videotape brief presentations and/or 
demonstration lessons and analyze their performances with the guidance of an instructor. Questions are asked and 
assignments are submitted via email. Some assignments are also burned to CD-RW and turned-in. Instructors 
correct the assignments on the CD-RWs and return them to the candidates. A portion of candidate research is done 
via the Internet and lesson plans are developed which incorporate a wide variety of technologies - from graphing 
calculators to e-pals. As a terminal activity in their “Block IF’ coursework (which incorporates EDU 395, EDU 359, 
and several other courses), candidates create showcase portfolios which incorporate, for example, their teaching 
philosophy, interdisciplinary lesson plans, and resume. Candidates can then use these portfolios as they apply for 
teaching jobs after student teaching. Einally, candidates are introduced to a variety of adaptive technologies and 
learn about them in tandem with Individual Education Plans (lEPs). 

EDU 395 and EDU 359 are entwined and co-taught in order to better facilitate NCATE (2001) standards for 
the candidates. The NCATE standards. International Society for Technology in Education (ISTE) National 
Educational Technology Standards (NETS) and Performance Indicators (ISTE, 2000), National Council for the Social 
Studies program standards (NCSS, 1997), National Council for Teachers of Mathematics program standards (NCTM, 
1998), National Science Teachers Association program standards (NSTA, 1998), and the National Council for 
Teachers of English program standards (NCTE, 1997) that the two courses attempt to address together are shown in 
Table 1 below. 

Table 1 

NCATE Standards Addressed in EDU 395 and EDU 359 



Technology Standards 


Secondary Education Standards 


NCATE standards with 
expectations for knowledge and 
use of technology 


ISTE NETS and Performance 
Indicators 


NCSS, NCTM, NSTA, NCTE 
Standards 


Standard I.C.l Content Studies 
for Initial Teacher Preparation 


III. A Eacilitate technology-enhanced 
experiences that address content 
standards and student technology 
standards. 


NCSS #1. Provide substantial 
instruction in academic areas within 
the social studies field. 




III.B Use technology to support 
learner-centered strategies that address 
the diverse needs of students. 


NCTM#1. Identify, teach, and model 
problem solving. 

NSTA#8. Use prior conceptions and 
student interests to promote learning. 




III.C Apply technology to develop 
students’ higher-order skills and 
creativity. 


NSTA#1. Structure and interpret 
concepts, ideas, and relationships 
that are defined by state and national 
standards. 






NSTA#2. Know and understand 
scientific inquiry. 






NCTE#7. Engage students in 
discussion for the purposes of 
interpreting and evaluating ideas 
presented through oral, written, and 
visual forms. 




III.D Manage student learning 
activities in a technology-enhanced 
environment. 


NCTE#9. Use instruction that 
promotes understanding of varied 
uses and purposes of content. 
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Standard I.D.2 Professional and 
Pedagogical Studies 



V.A Apply technology in assessing 
student learning of subject matter using 

a variety of assessment techniques. 

II. A Design developmentally NCSS#2. Implement a course or 

appropriate learning opportunities that courses in the teaching of social 
apply technology-enhanced studies, including the nature of the 

instructional strategies to support the discipline and ideas, strategies, and 
diverse needs of learners. techniques for teaching social 

studies. 

NSTA#5. Use diverse and effective 
actions, strategies, and 

methodologies to teach. 

II. B Apply current research on teaching NCTM#2. Use a variety of physical 
and learning with technology when and visual materials for exploration 

planning learning environments and and development of mathematical 

experiences. concepts in grades 7-12. 

NCTM#5. Use instructional strategies 
based on current research as well as 
national, state, and local standards 

relating to mathematics instruction. 

II. E Plan strategies to manage student NCTE#3. Organize classroom 
learning in a technology-enhanced environment and learning experiences 

environment. that promote effective whole class, 

small group, and individual work. 

NCTM#4. Know when and how to 
use student groupings such as 
collaborative groups, cooperative 
learning, and peer teaching. 

NSTA#6. Organize and manage 
activities in different student 

groupings. 

IV. B Use technology resources to NSTA#7. Use advanced technology, 

collect and analyze data, interpret 
results, and communicate findings to 
improve instructional practice and 

maximize student learning. 

IV. C Apply multiple methods of NCTE#2. Design instruction to meet 

evaluation to determine students’ the needs of all students and provide 

appropriate use of technology for students’ continuous progress 

resources for learning, communication, and success, 

and productivity. 

NCTE#6. Incorporate technology and 
print/non-print media into instruction. 

NCTE#11. Provide students with 
appropriate reading strategies that 
permit access to and understanding 
of a wide range of print and non-print 
texts. 
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Standard IV. B Resources for 
Teaching and Scholarship 



V.B Continually evaluate and reflect on NCSS#3. Interact with exemplary 
professional practice to make informed teachers and scholars in the field of 
decisions regarding the use of social studies and social studies 

technology in support of learning. education. 

NCSS#4. Have multiple clinical 
experiences in social studies 
classrooms and supervision by 
qualified social studies education 

professionals. 

V.C Apply technology to increase 
productivity. 

V. D Use technology to communicate NCTE#10. Engage students in making 
and collaborate with peers, parents, and meaning of texts through personal 
the larger community in order to nurture response. 

student learning. 

VI. A Model and teach legal and ethical 
practice related to computers. 

VLB Apply technology resources to 
enable and empower learners with 
diverse backgrounds, characteristics, 

and abilities. 

LA Demonstrate introductory NCTE#1. Examine, evaluate, and 

knowledge, skills and understanding of select resources, such as textbooks, 
concepts related to technology. other print materials, video, film, 

recordings, and software that support 

the teaching of English language arts. 

LB Demonstrate continual growth in 
technology knowledge and skills to 
stay abreast of current and emerging 

technologies. 

II.C Identify and locate technology NCTM#6. Work on an 

resources and evaluate them for interdisciplinary team and in an 

accuracy and suitability. interdisciplinary environment. 

NCTM#7. Introduce and involve 
perspective teachers to the 
professional community of 

mathematics educators. 

II. D Plan for the management of NSTA#13. Use community, human 

technology resources within the context and institutional resources to 
of learning activities. advance the learning of science in the 

classroom and field. 

NSTA#21. Work willingly with peers, 
supervisors, and others in a 

professional manner. 

V.A Use technology resources to NCTE# 4. Develop interdisciplinary 

engage in ongoing professional teaching. 

development and lifelong learning. 
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VI. C Identify and use technology 
resources that affirm diversity. 


NSTA#18. Know and participate in 
professional organizations and 
activities of the science education 
community. 


VI.D Promote safe and healthy use of 
technology resources. 


NCTE#5. Create learning 
environments, which promote respect 
for, and support of individual 
differences of ethnicity, race, 
language, culture, gender, and ability. 


VI.E Eacilitate equitable access to 
technology resources for all students. 





Technology, Teaching, and learning 

Technology supports teaching, learning, and the classroom environment through specific techniques and 
procedures. In the area of teaching, technology encourages educators to remain current with trends and 
developments in learning theories and practices through on-line lesson plans. Teachers, even in isolated areas, can 
participate in chats and forums, and obtain information on conferences or competitions. Professional growth is 
possible through on-line journals. Learning is enhanced when students are able to explore beyond their own sphere 
of influence through such activities as internet-based research projects. Class presentations can incorporate video, 
audio, clip art and downloaded images from museums, galleries, or historic sights. 

In the area of learning, technology provides opportunities to develop critical thinking skills through 
evaluation of web sites. Students can gain experience in problem solving by employing a variety of resources, for 
instance utilizing a drawing program to design an invention that resolves a challenge. Likewise, students can 
communicate instantaneously across the country or around the world, i.e. sending email to the scientists working in 
Antarctica. Using maps online, students can create a cross-country trip to visit national parks and historic sites. The 
unit could include math concepts such as mileage, a photo journal using pictures imported from web sites, a word 
processing diary, and a science component that incorporated investigation of geology, meteorology, or botany along 
the way. Social studies could follow a bill through the legislature by reviewing the Congressional Record (2002) 
each day via the Internet. As a culmination of these activities, students could present a collaborative interdisciplinary 
project by locating primary source documents, historic photographs, recordings, statistical data, and antique maps 
from the Library of Congress’ American Memory project (2002). 

In the area of classroom management, technology can be employed to streamline routine activities such as a 
seating chart, attendance reports or the recording of grades. Moving students’ seating is just a “cut and paste” 
away. Specific software programs both document and analyze such data as absence and tardy records thus 
discouraging truancy and selective absenteeism. Grading programs offer full class scatter plots or an individual 
student’s success presented in graph form. These techniques are all methods to assist educators in determining 
areas for remediation or enrichment. 



Technology and Assessment 

While the Plattsburgh State University Adolescence/Health Education program is currently infusing 
technology into teaching practices and learning strategies, it has also been one of the forerunners in technology- 
enhanced programmatic assessment. As part of the degree in Adolescence/Health Education, students are required 
to create a digital portfolio showcasing much of their work from the various courses in their program. Students may 
include such items as a statement of educational philosophy, interdisciplinary lesson plans, resume and contact 
information, letter of introduction, and links to useful resources. The portfolio may also include digital pictures or 
video of the candidates and is (usually) created in Microsoft ErontPage and uploaded to the Plattsburgh State 
University student web server. This activity proves both enjoyable and challenging to candidates as many have 
never created a website before and for those that have, this project is much more in-depth and time-consuming. The 
intent is that the web-portfolios are to be viewed by potential employers so a good “electronic impression” is 
essential. Additionally, the activity reinforces to candidates the importance of saving, refining, and reflecting upon 
many assignments in their various education courses. It is impossible to successfully complete the portfolio without 
incorporating work from previous classes. Moreover, by incorporating these works into their web portfolio, it gives 
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candidates the opportunity to visit the previously completed work and revise or refine it. Certainly, most candidates 
reflect on their previous thoughts and writings as they are incorporating them into their digital portfolios. 

As well being assessed via an electronic format, candidates are also instructed on ways to assess 9th-12th 
grade students who participate in technology-enhanced lessons. Candidates are taught how to use portfolio -based 
assessment and grading rubrics with their own future students. While various types of testing and grading 
methodologies are covered in EDU 395 and EDU 359, candidates soon realize that technology-infused lessons lend 
themselves best to project-oriented activities and rubric and/or portfolio/presentation style assessments. These 
types of assessments also fall within the sorts of assessment activities encouraged by NCATE and important in the 
current educational climate. 



Challenges 

While the combination of the technology and curriculum courses certainly added breadth and richness to 
the overall Adolescence/Health Education program, it was not without its challenges. One of the primary difficulties 
of successfully combining any two courses is in the planning. Units and individual lessons need to be fleshed-out 
and coordinated well in advance of a normal preparatory timeline. In the life of university faculty, finding time to 
collaborate on each and every lesson taught within a course is difficult. Often, as in this case, more than one 
planning session for any lesson needed to occur as resources were obtained and materials created. Most faculty do 
not plan lessons weeks in advance and keeping a rigorous planning schedule can be quite taxing and time- 
consuming. In our estimation it is worth the benefits, but it is also important to be aware that extra planning and 
collaboration time must be accounted for. 

Similarly, for faculty to successfully combine courses, a bit of education must occur on the part of the 
course instructors. While the technology teacher and the curriculum teacher each had a good amount of knowledge 
of the other’s area, neither were experts in both fields. Therefore each needed to take time and explain to the other 
methodologies and tenets within the two fields in order for the course collaborations to make sense. This sharing 
was extremely valuable from a professional development standpoint and it would behoove educational institutions to 
consider co-teaching as a form of professional development for truly, it is. 

Other challenges that we faced in this endeavor were regarding lessons that simply didn’t mix the concepts 
or objectives of the two courses well. It is difficult to find a way to enhance every lesson with technology and to be 
sure, not every lesson should incorporate technology. However, what initially seemed to us to be a frustration 
actually became quite an effective teaching tool. Emphasizing through modeling and practice the types of lessons 
that did not lend themselves well to technology improved the students’ learning of these issues. 

Lastly, while ordinarily descriptions of technology experiences include occurrences of “when the 
technology fails’’ as a challenge or aggravation, we choose not to do that. In fact, we think that failing technology in 
a class of preservice teacher candidates is a “teachable moment.’’ Too often when teacher education candidates are 
taught about infusing technology into their teaching practices it is in a well-equipped university computer lab with 
media maintenance people only a phone call or office away. This is clearly not what these candidates will encounter 
when they become teachers in public or private K-12 schools. Very often the technology will not be up-to-date nor 
easily repaired in the K-12 setting so the opportunity for the “technology to fail’’ is very real. It is better for faculty to 
have that happen in a university class so that trouble-shooting techniques and classroom management skills can be 
modeled and discussed. Tailing technology is a very real part of a K-12 teachers professional life and should 
therefore be addressed in their professional training in a healthy, competent manner. 

Future Directions 

One critical area that our practice has not addressed is how technology can be used to enhance teaching 
and learning in content specific pedagogical techniques. Part of the reason for this is that content area expertise at 
the secondary level is quite advanced and is taught by faculty outside of education programs at Plattsburgh State 
University. While liberal arts faculty educate candidates in their subject matter, education faculty then cover literacy 
education, educational philosophy and theory, and curriculum is sues. Although this system encompasses much of 
what future teachers need in their repertoire prior to student teaching, one critical area that is not sufficiently 
addressed is the specific pedagogy used to teach a content area or specific subject. This is because it is difficult to 
find faculty who have advanced expertise in a subject matter (i.e. physics, calculus) as well as the pedagogical 
techniques that facilitate knowledge and skill understanding and acquisition. Currently at Plattsburgh State, a course 
entitled EDU 420, Seminar in Content Specific Methodology, endeavors to address this issue. The course, which is 
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fifteen hours in length, covers standards, content-driven pedagogy, and assessment theory within each of the four 
major disciplines: English, math, science, and social studies. As well, the course provides an opportunity for 
candidates to observe a secondary classroom teacher in action. Our concern at present regarding this course is 
twofold: 1) There is little consistency among the cooperating teachers regarding objectives, activities, and 
assessment for this course. 2) Fifteen hours, of which six is solely for observation, is insufficient to provide pre- 
service teachers with adequate content and opportunities for practice. However, at this time specialized faculty are 
being sought in order to strengthen the Adolescence/Health Education program and comply with the NCATE 
accreditation process. Once this faculty is identified, only then can the area of technology-enhanced content area 
pedagogical techniques be comprehensively addressed. 

Finally, while combining Plattsburgh State University’s EDU 395, “Curriculum and Instruction for Secondary 
Schools,’’ and EDU 359, “Integrating Technology into Teaching Practices’’ in order to improve preservice teacher 
education candidates and address NCATE standards was not without it’s trials, the overall endeavor was successful 
and continually evolves to improve. 
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